Quantitative diagnosis method, device and equipment for psychological abnormal diseases

Abstract

The invention relates to a quantitative diagnosis method, device and equipment for psychological abnormal diseases, and relates to the technical field of large language models.The method comprises the steps that in the interrogation interaction process with a target object, an input voice set of the target object is continuously obtained, and each piece of voice information in the input voice set is obtained; the method comprises the following steps of: performing care type common-situation output based on a care type reply requirement through a psychologist model, extracting a scale question related to evaluation from a preset other-evaluation mental scale through a preset psychologist model, and obtaining a question and answer data pair set according to a question question of a previous round of dialogue and an input voice set, and performing multi-modal quantitative scoring processing on the set according to the question and answer data to obtain an inquiry result of the target object after multiple rounds of cumulative scoring. Visibly, the analysis and quantification are carried out by combining the scale and the diagnosis dialogue data of the patient to extract the key emotion index, so that rapid diagnosis and screening of psychological abnormal diseases such as depression can be objectively, accurately and efficiently assisted.

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a quantitative diagnosis method, device and equipment for abnormal psychological diseases. Background Art

[0003] At present, the existing mental health assessment and psychological abnormality disease assessment schemes mainly rely on outpatient doctors' online and offline consultations. However, the traditional manual consultation of doctors is inefficient, expensive, and resource-scarce, which is not conducive to the large-scale screening of mental illnesses. In addition, the existing schemes also have analysis methods based on biological signals combined with information technology. Specifically, neural networks or random forest algorithms in computer technology are used to combine EEG signals, metabolites, mitochondria, sound timing signals, image signals, and text data from social media for analysis and diagnosis. However, this scheme has major problems such as low diagnostic accuracy and poor interpretability. It is vague or non-specific. This ambiguity limits its effectiveness and actual accuracy in practical applications, resulting in it being unable to truly become a clinically recognized analytical and diagnostic method. It can be seen that the existing mental health assessment and psychological abnormality disease assessment schemes have problems such as high resource consumption, low diagnostic efficiency, and low diagnostic accuracy. Summary of the invention

[0004] The present application provides a quantitative diagnostic method, device and equipment for abnormal psychological diseases. By combining scale analysis with the patient's diagnostic dialogue data to extract key semantic features and emotional indicators, it achieves objective, accurate and efficient auxiliary rapid diagnostic screening of depression, and solves the problems of high resource consumption, low diagnostic efficiency and low diagnostic accuracy in the prior art.

[0005] In a first aspect, the present application provides a quantitative diagnosis method for abnormal psychological diseases, comprising:

[0006] During the consultation interaction process with the target object, continuously obtaining an input speech set of the target object, wherein the input speech set includes at least one piece of speech information;

[0007] For each voice message in the input voice set, a caring empathy output is performed based on the requirement of a caring reply through the psychologist model, and evaluation-related scale questions are extracted from a preset other-assessment mental scale through the preset psychologist model;

[0008] Obtaining a question-answer data pair set according to the question asked in the previous round of dialogue and the input speech set, wherein the question-answer data pair set includes at least one question-answer data pair;

[0009] Based on the question-and-answer data, the set is processed with multimodal quantitative scoring to obtain the diagnosis results of the target object after multiple rounds of cumulative scoring.

[0010] Optionally, for each voice message in the input voice set, performing caring empathy output based on the requirement of caring reply through the psychologist model includes:

[0011] Perform text conversion based on the voice information to obtain voice text;

[0012] According to the requirements of caring response, the psychologist model determines the empathic response voice based on the voice text;

[0013] Constructing a digital human video for the empathic response voice;

[0014] In the real-time communication with the target object, the empathic response voice is output through the digital human video to provide caring empathic output to the target object.

[0015] Optionally, the step of extracting assessment-related scale questions from a preset other-assessment mental scale using a preset psychiatrist model includes:

[0016] Extract the other-rated mental scale from the preset scale database;

[0017] With respect to the voice information, assessment-related scale questions are extracted from the other-rated mental scale.

[0018] Optionally, performing multimodal quantitative scoring processing on the set based on the question-answer data to obtain the diagnosis result of the target object after multiple rounds of cumulative scoring includes:

[0019] Performing sentiment semantic analysis based on the question-answer data pair set to obtain semantic key information, wherein the semantic key information includes a key question-answer data pair set;

[0020] Using the psychologist model, the set is quantitatively scored according to the key question and answer data to obtain a multimodal quantitative score;

[0021] Performing mapping diagnosis according to the multimodal quantitative score to obtain diagnostic information, wherein the diagnostic information is used to characterize the symptom condition of the target object;

[0022] Visualization processing is performed based on the diagnostic information to obtain a consultation result of the target object, wherein the consultation result includes a visualized consultation report diagram.

[0023] Optionally, the psychologist model is used to quantitatively score the set according to the key question and answer data to obtain a multimodal quantitative score, including:

[0024] When performing caring empathy output to the target object, acquiring expression video data and audio data of the target object;

[0025] Performing emotion dimension recognition based on the expression video data to obtain image dimension information, and performing emotion dimension recognition on the audio data to obtain voice dimension information;

[0026] Performing topic analysis on the set according to the question-answer data to obtain scoring dimension information;

[0027] Based on the image dimension information, weighted processing is performed in combination with the speech dimension information and the scoring dimension information to obtain a multimodal quantitative score.

[0028] Optionally, performing sentiment semantic analysis on the question-answer data set to obtain semantic key information includes:

[0029] Performing data conversion on the set of question-answer data to obtain discrete data;

[0030] According to the preset segmentation rules, the discrete data is subjected to sentiment semantic segmentation to obtain semantic key information.

[0031] Optionally, before extracting the caring reply information corresponding to the voice information from a preset evaluation table using a preset psychologist model, the method further includes:

[0032] Obtaining sample question-answer data pairs, and constructing a sample data set based on the sample question-answer data pairs;

[0033] Building an initial model according to a preset model building component, wherein the model building component includes a multi-head attention mechanism, a relative position encoding, and a feedforward neural network;

[0034] Performing model training on the initial model based on the sample data set to obtain an initial large language model;

[0035] Performing loss parameter adjustment processing according to the initial large language model to obtain a target large language model;

[0036] The target large language model is used in combination with a preset expert knowledge base to construct a model to obtain the psychologist model.

[0037] Optionally, performing loss parameter adjustment processing according to the initial large language model to obtain a target large language model includes:

[0038] Determining a specific loss function and a fine-tuning loss function of the initial large language model, and constructing an additional network layer in the initial large language model based on the fine-tuning loss function;

[0039] Calculate the gradient of the specific loss function to the initial large language model through a preset back-propagation algorithm to obtain a first update parameter;

[0040] Based on the back propagation algorithm and the specific loss function, calculating the gradient of the fine-tuning loss function with respect to the additional network layer to obtain a second update parameter;

[0041] The initial large language model is updated with parameters according to the first update parameters, and the additional network layer is fine-tuned with parameters according to the second update parameters to obtain the target large language model.

[0042] In a second aspect, the present application provides a quantitative diagnostic device for abnormal psychological diseases, comprising:

[0043] An acquisition module, used for continuously acquiring an input speech set of the target object during the consultation interaction process with the target object, wherein the input speech set includes at least one piece of speech information;

[0044] an empathy output module, for performing caring empathy output based on the requirement of caring response through a psychologist model for each voice message in the input voice set, and extracting evaluation-related scale questions from a preset other-assessment mental scale through a preset psychologist model;

[0045] A question-answer data pair set determination module, configured to obtain a question-answer data pair set according to the questions asked in the previous round of dialogue and the input speech set, wherein the question-answer data pair set includes at least one question-answer data pair;

[0046] The multimodal quantitative processing module is used to perform multimodal quantitative scoring processing on the set based on the question and answer data, and obtain the diagnosis results of the target object after multiple rounds of cumulative scoring.

[0047] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0048] Memory, used to store computer programs;

[0049] The processor is used to implement the steps of the quantitative diagnosis method of abnormal psychological diseases as described in any embodiment of the first aspect when executing the program stored in the memory.

[0050] In summary, in the process of interactive consultation with the target object, the embodiment of the present application continuously obtains the input voice set of the target object, and for each voice message in the input voice set, the psychologist model performs caring empathy output based on the requirements of caring replies, and extracts evaluation-related scale questions from the preset other-rated mental scale through the preset psychologist model, obtains a question-and-answer data pair set based on the questions asked in the previous round of dialogue and the input voice set, and performs multimodal quantitative scoring processing based on the question-and-answer data pair set to obtain the consultation results of the target object after multiple rounds of cumulative scoring. It can be seen that the present application establishes a complete, accurate, and efficient diagnosis system for depression and other psychological diseases, and analyzes and quantifies the diagnostic dialogue data of the patient by combining the scale with the patient to extract key emotional indicators, thereby realizing objective, accurate, and efficient rapid diagnosis and screening of auxiliary depression, which can effectively assist doctors in the early diagnosis and intervention of psychological abnormalities such as depression, and solve the problems of high resource consumption, low diagnostic efficiency, and low diagnostic accuracy in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 A flowchart of a quantitative diagnosis method for abnormal psychological diseases provided in an embodiment of the present application;

[0054] Figure 2 This is a schematic diagram of the steps of a quantitative diagnosis method for abnormal psychological diseases provided by an optional embodiment of the present application;

[0055] Figure 3 Is a table diagram of questions and topics of the Hamilton Depression Screening Scale provided by an optional example of this application;

[0056] Figure 4 Is a table diagram of supplementary questions for the Hamilton Depression Screening Scale provided by an optional example of this application;

[0057] Figure 5 It is a structural block diagram of an automatic voice and text conversion unit combined with a digital human video provided by an optional example of the present application;

[0058] Figure 6This is a schematic diagram of a process setting for a flexible dialogue and guiding users to answer questions provided by an optional example of this application;

[0059] Figure 7 This is an optional example of a basic consultation unit and a dialogue example diagram provided by this application;

[0060] Figure 8 This is a flowchart of a Tokenizer provided as an optional example of the present application for processing input data;

[0061] Fig. 9 It is a schematic diagram of the scoring process of a quantitative scoring unit of a large model of a psychologist provided by an optional example of this application;

[0062] Fig.10 A structural block diagram of a quantitative diagnosis device for abnormal psychological diseases provided in an embodiment of the present application;

[0063] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0065] To facilitate the understanding of the embodiments of the present application, further explanation will be given below in conjunction with the drawings and specific embodiments. The embodiments do not constitute a limitation on the embodiments of the present application.

[0066] Figure 1 A flowchart of a quantitative diagnosis method for abnormal psychological diseases provided in an embodiment of the present application. Figure 1 As shown, the quantitative diagnosis method for abnormal psychological diseases provided in the embodiment of the present application may specifically include the following steps:

[0067] Step 110: continuously obtain an input speech set of the target object during the consultation interaction process with the target object.

[0068] Wherein, the input voice set includes at least one voice information.

[0069] In this embodiment, the target object includes a user, and the user may include but is not limited to a patient who is undergoing a consultation, usually a person who needs to undergo a diagnosis of a psychological disorder or mental health, which is not limited in this embodiment. The voice information may include but is not limited to: questions asked by the user and / or the content of the user's answer, etc.

[0070] In a specific implementation, this embodiment can pre-build a digital human interaction interface and use the digital human interaction interface to conduct medical consultation activities. Specifically, through the digital human interaction interface, this embodiment can continuously obtain the user's voice information during the medical consultation interaction process with the user. During the medical consultation process, the professional psychologist model (also known as the psychologist model, the psychologist large model, the large language model, etc.) in this embodiment sequentially conducts multiple inquiries to the user from the doctor's perspective, which may include but not be limited to questions and empathetic replies, etc. The user's voice information may include but not be limited to the user's voice reply to each inquiry and the user's description of himself or the current situation, etc., and the input voice set is formed by aggregating multiple voice information.

[0071] It should be noted that the quantitative diagnosis method for abnormal psychological diseases provided in this application involves technical application fields such as smart medical care, mental health assessment, and large language models, and can be applied to the quantitative diagnosis of various psychological abnormalities, including but not limited to: depression, mania and other abnormal psychological diseases.

[0072] Step 120, for each voice message in the input voice set, a caring empathy output is performed based on the requirement of a caring response through the psychologist model, and assessment-related scale questions are extracted from a preset peer-assessed mental scale through the preset psychologist model.

[0073] In this embodiment, the peer-assessment mental scale (or assessment scale, peer-assessment scale, etc.) may include at least one diagnostic assessment scale used mainly in the medical consultation process. The diagnostic assessment scale may include but is not limited to the Hamilton Depression Rating Scale (HAMD) commonly used for depression, the Young Mania Rating Scale (YMRS) commonly used for mania, and other clinical-level peer-assessment scales for psychological disorders.

[0074] In this embodiment, the scale questions can be questions for subsequent consultation extracted based on the user's input voice, and this embodiment does not limit this. The psychologist model can extract scale questions for evaluation from the other mental scale based on the user's input voice.

[0075] In the specific implementation, the psychologist model can have good language, semantic analysis and semantic understanding capabilities. During the entire communication process with the user, it can continuously extract key information based on the user's questions and answers to understand the user's current emotions. For each voice message in the input voice set, the caring reply information is determined according to the requirements of the caring reply. The caring reply information is used to output caring empathy, and a targeted question for diagnosis is read / selected / extracted from the other-rated mental scale according to the input voice as the scale question. The caring reply information and the scale question can then be combined to form a two-stage reply. The psychologist model can use the digital human interactive interface, use the digital human as an information carrier to interact through a good visual experience and a gentle voice, and output the two-stage reply to the user, thereby realizing a caring empathy reply first for the user's input voice, and then conduct a diagnosis, thereby improving the user experience.

[0076] In actual implementation, this embodiment can collect multiple other-rated mental scales in advance, and build a scale database based on the collected other-rated mental scales. In the process of interactive consultation with the user, the scale questions can be extracted from the database according to the user's voice input according to certain strategies or consultation needs, and combined with empathic replies to form a two-stage reply.

[0077] In the related art, the existing mental health assessment and psychological abnormal disease assessment schemes also use mental health assessment applications (referred to as APP) to use the user's mental health self-assessment scale for self-assessment, that is, a method of evaluating psychological abnormal diseases through self-assessment, such as the common PHQ-9 self-assessment scale for depression, the SAS self-assessment scale for anxiety, etc. However, the scoring items of the user's self-assessment scale and the scoring of the self-assessment items are usually given by the user based on his or her subjective feelings and understanding of the state, which is very subjective. The self-assessment depends to a large extent on the patient's understanding of himself or herself and the subjective willingness to accept psychological screening. It can be seen that the existing method of evaluating psychological abnormal diseases through self-assessment has major drawbacks and cannot truly become a clinically recognized analytical and diagnostic method. To solve this problem, this application obtains one or more other-assessed mental scales according to actual needs. In addition to involving conventional psychological screening issues, different other-assessed mental scales can also have some evaluation items from the doctor's perspective that are different from the user's self-assessment scale. For example, the Hamilton Depression Rating Scale requires the doctor to give a score of the patient's anxiety level, depression level, and self-insight level during the consultation process based on the doctor's observation; the Young Mania Rating Scale requires the doctor to give a score of the patient's cooperation, aggressiveness, impatience, delusion level, etc. based on the process of the interview.

[0078] In this embodiment, the psychologist model can use good language and semantic analysis capabilities to keenly capture the patient's emotions and state changes during the chat consultation process for subsequent multimodal processing, which will not be described in detail in this application. Compared with the existing method of evaluating abnormal psychological diseases through self-evaluation, the results of scale screening can be given more objectively. At the same time, the doctor gives the patient's evaluation score based on the patient's performance during the consultation process to supplement the comprehensive score. This is the reason why the self-evaluation mental scale has more objectivity than the self-evaluation scale and is clinically accepted.

[0079] Step 130, obtaining a question-answer data pair set based on the questions asked in the previous round of dialogue and the input speech set.

[0080] The question-answer data pair set includes at least one question-answer data pair.

[0081] In this embodiment, by converting the user's input voice into corresponding text data, the good semantic understanding and analysis capabilities of the psychologist model are utilized to perform semantic analysis on the text data and extract key information. Based on the extracted key information, the psychologist model can extract scale questions to form a two-segment response for output while providing caring empathy output when responding to the user. Since during the consultation process, the user's input voice is usually the answer to the previous round of dialogue, which corresponds to the scale question output by the psychologist model during the previous round of dialogue. Therefore, in actual implementation, this embodiment can combine the questions and input voice information of the previous round of dialogue to form a <question-answer> data pair, referred to as a question-answer data pair. Subsequently, all question-answer data pairs in the consultation process are aggregated to obtain a question-answer data pair set.

[0082] Step 140 , performing multimodal quantitative scoring processing on the set based on the question and answer data, and obtaining the diagnosis result of the target object after multiple rounds of cumulative scoring.

[0083] In this embodiment, each question in the other-rated mental scale and the user's answer to the question can be given a corresponding preset score. This embodiment can perform multimodal quantitative scoring based on the question and answer data set, including but not limited to: emotional semantic analysis and quantitative scoring, etc., by performing multiple rounds of cumulative scoring on the question and answer data set, thereby obtaining the user's final score, that is, the multimodal quantitative scoring. Subsequently, the user's diagnosis result can be obtained based on the multimodal quantitative scoring and combined with the other-rated mental scale. Taking depression as an example, the user's diagnosis result may include but not be limited to: normal, possible depression, depression, and severe depression, etc. Finally, the user's consultation result is obtained by combining the entire consultation process, the user's multimodal quantitative scoring, and the diagnosis result.

[0084] In actual implementation, for each question-answer data pair in the question-answer data pair set, this embodiment can perform multimodal quantitative scoring including semantic analysis for each question-answer data pair, and extract key information from the conversation process with the user by performing semantic analysis on the question-answer data pair, including but not limited to semantic understanding, semantic analysis, and keyword extraction. For topic conversation data with a certain elastic conversation process, the semantics of sentences related to the question can be accurately extracted, providing an accurate and objective basis for the quantitative scoring and consultation results of the psychologist model, and improving the interpretability of the quantitative diagnosis of abnormal psychological diseases.

[0085] For example, the last round of question-answer pairs can be: <Doctor asks: Have you had any changes in your meal size and weight recently? Patient answers: I have no interest in anything recently. I just want to be alone and don’t want to see anyone. However, there hasn’t been much change in my diet, and my weight is relatively stable>. The psychologist model performs semantic understanding and analysis on the question-answer data pairs, and extracts the corresponding keywords to obtain the key question-answer data pairs: <Doctor asks: Have you had any changes in your meal size and weight recently? Patient answers: There hasn’t been much change in my diet, and my weight is relatively stable>.

[0086] Exemplarily, for multimodal quantitative scoring, taking the Hamilton Depression Rating Scale as an example, the "depression level" scoring item in the Hamilton Depression Rating Scale can be: 0 points - no depression; 1 point - only expressed when asked; 2 points - in addition to the questions asked by the doctor, spontaneously describe more of their own depression; 3 points - extremely obvious depression is revealed in speech, and this emotion can even be revealed in expression, posture, voice or crying; 4 points - the patient's spontaneous speech and non-verbal expression (expression, action) almost completely express this emotion, and this example is not limited to this.

[0087] In actual implementation, quantitative scoring through multiple dimensions can more accurately and objectively simulate the process of human psychologists' consultation, thereby providing more objective quantitative scoring results. Specifically, multiple scoring dimensions / items can be preset, and corresponding weights can be preset for each scoring dimension / item. Weighted processing is performed based on the target object in each scoring dimension / item, combined with the corresponding weights, to obtain a multimodal quantitative score of the target object. This application does not elaborate on multimodal quantization here.

[0088] Therefore, the present application uses the large language model in the auxiliary diagnosis of mental health screening, and has completed the specific design and implementation route for the quantitative evaluation technology of abnormal psychological symptoms, including but not limited to: standardized process of medical consultation, empathetic output, emotional semantic analysis and quantitative scoring, etc. By analyzing the scale diagnosis dialogue data of users / patients in the real world, key semantic features and emotional indicators can be extracted, thereby providing an objective, accurate and efficient method for auxiliary rapid diagnosis and screening of abnormal psychological diseases such as depression, and establishing a complete, accurate and efficient diagnosis system for depression and other psychological diseases to assist doctors in early diagnosis and intervention of psychological diseases such as depression. Therefore, the embodiment of the present application improves the accuracy of diagnosis. Compared with the existing method of quantitative evaluation using model AI, which has ambiguity or lack of specificity in description, and the problem of limiting its effect and actual accuracy in practical application, the embodiment of the present application improves the quantitative evaluation method of the large model and solves the problems of ambiguity and lack of specificity in the prior art.

[0089] It can be seen that in the process of interactive consultation with the target object, the embodiment of the present application continuously obtains the input voice set of the target object, and for each voice message in the input voice set, a caring empathy output is performed based on the requirement of caring reply through the psychologist model, and evaluation-related scale questions are extracted from the preset other-assessment mental scale through the preset psychologist model, and a question-answer data pair set is obtained according to the questions asked in the previous round of dialogue and the input voice set, and a multimodal quantitative scoring process is performed on the question-answer data pair set to obtain the consultation result of the target object after multiple rounds of cumulative scoring, thereby analyzing and quantifying the diagnostic dialogue data of the scale and the patient to extract key semantic features and emotional indicators, thereby achieving objective, accurate and efficient auxiliary depression rapid diagnosis and screening, and solving various problems existing in existing mental health assessments and abnormal psychological disease assessment schemes.

[0090] Reference Figure 2 , shows a schematic flow chart of the steps of a quantitative diagnosis method for abnormal psychological diseases provided by an optional embodiment of the present application. The method may specifically include the following steps:

[0091] Step 210: continuously obtain the input speech set of the target object during the consultation interaction process with the target object.

[0092] Wherein, the input voice set includes at least one voice information.

[0093] For the description of step 210 , reference may be made to the description of step 110 , which will not be described in detail in this embodiment.

[0094] Step 220, for each voice message in the input voice set, a caring empathy output is performed based on the requirement of a caring response through the psychologist model.

[0095] For the description of step 220, reference may be made to the above step 120, which will not be described in detail in this embodiment.

[0096] In an optional embodiment, the embodiment of the present application performs caring empathy output based on the requirement of caring response through a psychologist model for each voice message in the input voice set, which may specifically include: performing text conversion based on the voice information to obtain a voice text; determining an empathy response voice based on the voice text through the psychologist model in accordance with the requirement of caring response; constructing a digital human video for the empathy response voice; and outputting the empathy response voice through the digital human video in real-time communication with the target object to perform caring empathy output for the target object.

[0097] In order to realize the interaction of voice consultation with the user, in actual implementation, this embodiment can construct an automatic conversion unit for voice and text, as well as a digital human. The psychologist model can combine the automatic conversion unit and the digital human, use the automatic conversion unit to complete the mutual conversion of voice and text, and use the digital human as an information carrier to interact with the user's voice information through a good visual experience and gentle voice, and output a caring and empathetic response to the user's input.

[0098] Specifically, for the automatic conversion unit of speech and text, this embodiment deploys the automatic conversion model of speech and text and api (application programming interface) service on the server to provide the functions of user speech to text and psychologist model text to speech. Further, the psychologist model can integrate the model VITs (Variational Inference with adversarial learning for end-to-end Text-to-Speech, speech synthesis method), VITs can provide text-to-speech function service for the psychologist model, and refit the model by providing a certain amount of speech data, so that the output speech timbre of the speech model is closer to the trained speech data, thereby achieving more vivid and comfortable customized digital human interaction needs. Thus, by using the automatic conversion unit of speech and text, the user's input speech set can be converted into speech text, which is convenient for the psychologist model to understand semantics, and the psychologist model can also determine empathy response information for speech text, convert the empathy response information into empathy response speech, and output empathy response speech through the digital human, so as to achieve caring empathy output for the user.

[0099] Step 230, extracting the other-evaluation mental scale from the preset scale database.

[0100] Step 240: extracting assessment-related scale questions from the other-assessment mental scale for the voice information.

[0101] A unified description of steps 230 to 240 is given as follows:

[0102] In this embodiment, the other-assessment mental scale can include a main question stream, a scoring indicator description (for subsequent multimodal quantitative scoring), and a comprehensive score conversion diagnosis result. By aggregating multiple scales, a scale database can be constructed. Among them, the main question stream is used for questions asked during the consultation process as scale questions for quantitative evaluation of users. The doctor's response during the consultation process consists of the caring and empathetic responses of the professional psychologist model plus the scale questions extracted by the main question stream according to a certain strategy.

[0103] Taking the Hamilton Depression Rating Scale as an example, the topics covered in the scale include: Figure 3 As shown in the figure, the additional scoring items from the doctor's perspective are as follows Figure 4 As shown. The scoring indicators follow the clinical description of the scale. For example, for the conversation data on the topic of "systemic symptoms", the following quantitative scoring description can be given: 0 points - no systemic pain symptoms; 1 point - heaviness in the limbs, back or neck, back pain, headache, muscle pain, general weakness or fatigue; 2 points - obvious pain in the limbs, back, neck or general weakness. The subsequent psychologist model can analyze the semantics of the conversation data of the patient's related topics during the conversation according to the quantitative scoring instructions and give a quantitative score. This example will not be described in detail.

[0104] During the consultation process, the clinical result conversion instructions of the quantitative score can be stored for use in the output of the final consultation result. For example, in the form of "score: consultation result", the quantification can be stored as follows: "0-6": "normal", "7-17": "possible depression", "18-24": "depression", ">24": "severe depression".

[0105] In actual implementation, the psychologist model can output scale questions through digital human video. Specifically, the psychologist model can combine the empathy response information used for caring empathy response and scale questions as a two-segment response, and convert the two-segment response into corresponding voice for output through an automatic conversion unit.

[0106] In the specific implementation, for digital human videos, this embodiment deploys a digital human real-time driving API service on the server, so that the psychologist model can call the digital human through the API service to generate digital human videos in real time, thereby realizing real-time driving of the digital human. The input end of the digital human service mainly comes from the text data generated by the large model and the static picture of the digital human doctor. The output end can correspond to the digital population type of the input text data to drive the video stream data in real time. Figure 5 As shown, the psychologist model can combine the digital human with the above-mentioned automatic conversion unit of voice and text, and construct a digital human doctor in real-time communication with the user, and can combine the voice data obtained by text conversion to obtain a digital human virtual doctor who can communicate with the patient in real time, as a digital human video, and output empathic reply voice through the digital human video. Preferably, in order to improve the accuracy of communication with the user, the real-time communication delay can be within 2 seconds (s).

[0107] In an optional implementation, during the interactive process of dialogue and consultation, if the user has doubts about the question asked or the semantics of the answer content is judged by the psychologist's big model as not answering the question well, then the conversation related to this question will enter the elastic dialogue process. In the preset elastic dialogue round x, if x is set to 3, the elastic dialogue process can have up to three additional rounds of dialogue. In the elastic dialogue process, the user's questions are answered, and the prompt engineering of the big model and the external knowledge base index are used to give the user a more positive interaction experience and guide the user to answer the questions asked in the scale. The above method ensures that the user understands the scale questions and gives the psychologist's big model enough semantic information in the answer for quantitative scoring. The relevant flowchart can be seen in Figure 6 shown.

[0108] Take the example of depression screening and the consultation process of using the Hamilton Depression Rating Scale to implement a large model quantitative evaluation method for mental illness. In the consultation process, the psychologist model can act as a psychological therapist, listening to and accompanying the user, and completing the depression screening in the conversation consultation process. First, a caring and empathetic response is given to the user's input, and then a scale question is read from the database and inserted, combined into a two-part response, and finally the digital human is used as an information carrier to interact through a good visual experience and a gentle voice. Furthermore, the basic consultation unit based on the psychologist model and the scale database can be achieved through the following methods: Figure 7 The basic consultation units and dialogue example diagrams shown are explained.

[0109] Furthermore, in order to solve the problems of high cost, low accuracy and poor interpretability of existing psychological abnormal disease evaluation and diagnosis methods, this embodiment provides a large language model AI quantitative evaluation technology implementation route and specific design in mental health screening auxiliary diagnosis, that is, a quantitative diagnosis method for psychological abnormal diseases is provided, including but not limited to: standardized process of consultation, data preprocessing, model selection, feature extraction, quantitative scoring and result interpretation and other specific steps and parameter settings. This implementation regards the mental health screening method based on the large model as a very important key technology improvement to achieve standardized AI quantitative evaluation of psychological abnormal diseases.

[0110] The large language model AI quantitative evaluation method in the embodiment of the present application may include but is not limited to: a pre-trained professional psychological large language model (simulating a doctor), a tokenizer, a scale database, a large model quantitative scoring calculation unit, a supplementary quantitative scoring unit from another evaluation perspective, a voice collection and sound generation device, a comprehensive diagnosis calculation unit, and a digital human interactive computer interface.

[0111] For the professional psychological big language model, that is, the psychologist model, this application makes innovative improvements based on the existing open source big language model. By fine-tuning the existing open source big language model, a professional big language model of a simulated doctor that can provide clinical standards is obtained, thereby facilitating the subsequent quantitative diagnosis of abnormal psychological diseases for users.

[0112] In an optional embodiment, before extracting the caring reply information corresponding to the voice information from a preset rating scale through a preset psychologist model, the method may further include: obtaining sample question and answer data pairs, and constructing a sample data set based on the sample question and answer data pairs; constructing an initial model according to a preset model construction component, wherein the model construction component includes a multi-head attention mechanism, a relative position encoding, and a feedforward neural network; performing model training on the initial model based on the sample data set to obtain an initial large language model; performing loss parameter adjustment processing based on the initial large language model to obtain a target large language model; and constructing a model based on the target large language model in combination with a preset expert knowledge base to obtain the psychologist model.

[0113] In this embodiment, the sample data set includes a publicly available depression clinic dialogue data set, which includes but is not limited to Chinese data sets such as D4 or Smile Chat, and the main components of the data set can be <dialogue-answer> data pairs between patients and doctors. The Chinese data set can be used to fine-tune the large language model to improve the model's Chinese recognition ability, semantic understanding and analysis ability, and dialogue ability, and then the data set is used to perform model self-supervision training on the large language model. During the model training process, the large language model is subjected to loss parameter adjustment processing using the defined loss function to continuously optimize the model. Finally, the model is constructed in combination with a preset expert knowledge base such as the langchain external expert knowledge base, thereby obtaining a trained psychologist model.

[0114] In the specific implementation, this embodiment can use the Transformer architecture as the basic structure of the large language model. The Transformer architecture is the core of the large language model. It relies on a multi-layer stack of self-attention mechanisms, which enables the model to weigh the importance of different words in a sentence when processing each word. This self-attention mechanism helps the model to effectively capture long-range dependencies in the text. Based on the Transformer architecture, this embodiment combines multiple model building components such as multi-head attention mechanism, relative position encoding, and feedforward neural network for the construction of a large model of psychologists.

[0115] Among them, the multi-head self-attention mechanism (i.e., multi-head attention mechanism) is used to enable the model to focus on the relevant parts of the input text while generating each word in the output. The multi-head self-attention mechanism can better capture contextual information by calculating the interactions and correlations between different positions in the input sequence. The calculation process can be expressed by the following formula (1):

[0116]

[0117] Among them, Q represents the query of the input sequence, K is the key of the input sequence, V is the value of the input sequence, and d_k represents the dimension of each attention head.

[0118] Relative position encoding means that Transformer uses relative position encoding to capture the relative distance between different positions in the input sequence. The calculation of relative position encoding can be expressed by the following formula (2):

[0119]

[0120] Among them, PE {(pos,2i)} Represents the relative position encoding of position pos and dimension 2i, PE {(pos,2i+1)} Represents the relative position encoding of position pos and dimension 2i+1. d_model represents the dimension of the model.

[0121] The feedforward neural network is used to perform nonlinear transformation and feature mapping on the hidden state of each position. The calculation process can be expressed by the following formula (3):

[0122] FFN(x)=max(0,xW_1+b_1)W_2+b_2 Formula (3)

[0123] Among them, x represents the hidden state of the input, W_1, W_2, b_1 and b_2 are all parameters of the model.

[0124] Using multi-head attention mechanism, relative position encoding and feedforward neural network as the core components in the Transformer structure, through relative position encoding, multi-head self-attention mechanism and feedforward neural network, Transformer can capture the long-range dependencies of sequence data in natural language processing tasks and generate accurate output.

[0125] Preferably, the embodiment of the present application can select a large model based on the Transformer structure, combined with the large model of LLaMa3-8B as the basic model, and on this basis, the empathy dialogue ability of the open source LLaMa3-8B large model in the psychological field can be improved through subsequent pre-training and fine-tuning training. Specifically, large models based on the Transformer structure, such as ChatGPT, can generate outputs similar to human language by learning language rules and patterns from a large amount of data, and even conduct high-quality multi-round dialogues. The Transformer structure directly captures the dependencies between different positions in the text sequence through the self-attention mechanism (Self-Attention), avoiding the gradient disappearance and gradient explosion problems that are prone to occur in traditional RNN and LSTM models when processing long sequences. This enables the Transformer model to better model long-term dependencies when processing long texts, while being able to consider information at all positions, thereby more comprehensively modeling contextual relationships. This global information modeling capability enables the Transformer architecture model to grasp the global semantics more accurately when understanding text, and thus can adapt to a variety of different task requirements, whether it is machine translation, text generation or text classification tasks, can be optimized by adjusting the structure of the encoder and decoder. On the other hand, although ChatGPT has strong performance and is easy to use through API calls, since openAI chooses not to open source the weight parameters and related codes of the ChatGPT model, it cannot well support the needs of model pre-training and instruction fine-tuning after collecting data. Therefore, the design concept of this application turns its attention to the more open and convenient open source model LLaMa for research and use. The large models of the LLaMa series are developed, trained and open sourced by Meta. The latest open source LLaMa3 series large models in 2024 use a corpus of up to 15Ttoken for pre-training. The huge data set covers a wide range of knowledge fields, allowing LLaMa3 to demonstrate excellent capabilities in generating text, understanding complex contexts and cross-domain tasks. At the same time, Meta has open-sourced large models with various parameter amounts such as 8B and 70B, further enhancing the high scalability of the model for use in different occasions. Therefore, in the application scenario where the computing resources of the deployed server are limited and the digital human and large model are required to respond quickly, this application selects the large model of LLaMa3-8B as the basic model, and on this basis, through subsequent pre-training and fine-tuning training, improves the empathy dialogue ability of the open source LLaMa3-8B large model in the psychological field.

[0126] In actual processing, the embodiment of the present application combines the LangChain external expert knowledge base to jointly form a large model of psychology, and promotes the integration of external knowledge bases of psychology through LangChain, further realizing knowledge enhancement and achieving professional empathy answering effects in the field of psychology. When the psychologist large model performs the user's answering task, it can dynamically query, understand and integrate knowledge information from multiple sources, including but not limited to: knowledge information from multiple sources such as professional databases, academic literature, knowledge bases, etc., thereby enhancing the ability and advantages of the large model in psychology. In addition, in order to achieve effective conversion of natural language queries and external knowledge base query languages, LangChain adopts complex query conversion and optimization technologies, including the application of natural language understanding (NLU) models, parsing the intent and structure of queries, and query rewriting technology, converting natural language queries into query statements suitable for specific knowledge bases, specifically involving natural language logic formalization, query optimization algorithms, semantic similarity calculations, etc. The external knowledge acquired in this embodiment can also be integrated with the knowledge representation inside the model to support more complex reasoning processes. LangChain realizes the effective integration and utilization of cross-modal and cross-domain knowledge by designing sophisticated knowledge fusion mechanisms, such as attention mechanism and memory network, thereby enhancing the reasoning ability and creativity of the psychologist model in this embodiment.

[0127] In an optional embodiment, the above-mentioned loss parameter adjustment processing based on the initial large language model to obtain the target large language model may further include: determining a specific loss function and a fine-tuning loss function of the initial large language model, and constructing an additional network layer in the initial large language model based on the fine-tuning loss function; calculating the gradient of the specific loss function with respect to the initial large language model through a preset back propagation algorithm to obtain a first update parameter; calculating the gradient of the fine-tuning loss function with respect to the additional network layer based on the back propagation algorithm and the specific loss function to obtain a second update parameter; updating the parameters of the initial large language model according to the first update parameters, and fine-tuning the parameters of the additional network layer according to the second update parameters to obtain the target large language model.

[0128] In the process of model training and fine-tuning, this embodiment can define a loss function for updating and optimizing the parameters of the model. Specifically, in the process of fine-tuning, this embodiment can define a loss function suitable for a specific task as a specific loss function, and can define a fine-tuning loss function for fine-tuning parameters with high efficiency.

[0129] The specific loss function measures the performance of the model on a specific task. The goal of this embodiment is to minimize the loss function. In general network fine-tuning, the commonly used loss function cross-entropy loss (Cross-Entropy Loss) can be expressed by the following formula (4):

[0130] L(θ)=-Σ[y*log(p)] Formula (4)

[0131] θ is the network's parameters, y is the label (usually a one-hot vector), and p is the probability predicted by the model. A specific loss function measures the performance of the model on the classification problem by calculating the cross entropy between the label and the model's prediction. By minimizing the cross entropy loss, the parameters can be gradually adjusted to make the model's predictions closer to the actual labels.

[0132] For back propagation and parameter updating, the core of fine-tuning in this embodiment is to calculate the gradient of the loss function to the model parameters through the back propagation algorithm, and use the optimizer to update the parameters of the model so that it gradually converges to better performance. Specifically, for each parameter θ, the gradient of the loss function can be calculated. To calculate the updated parameter value, as shown in the following formula (5):

[0133]

[0134] Among them, θ_old is the current parameter value, and learning_rate is the learning rate, which is used to control the step size or speed of parameter update. By calculating the gradient of the loss function The derivative information about the parameter θ_old can be obtained, and then it is multiplied by the learning rate learning_rate, and then the result is subtracted from the current parameter value θ_old to obtain the new parameter value θ_new. This update process is usually performed in each training iteration and repeated many times until the predetermined number of iterations is reached or the convergence condition is reached. By continuously adjusting the parameters in the direction of gradient descent, the model will gradually optimize and the value of the loss function will gradually decrease to improve the performance and accuracy of the model.

[0135] For the fine-tuning loss function, this embodiment uses the LoRA (Low-Rank Adaptation of Large Language Models, literally translated as low-rank adaptation of large language models) large language model parameter efficient fine-tuning method. The basic principle of LoRA is to freeze the pre-trained model weight parameters. When the original model parameters are frozen, additional network layers are added to the model and only the newly added network layer parameters are trained. Since the number of these newly added parameters is small, not only the cost of fine-tuning the model is significantly reduced, but also the effect similar to that of fine-tuning the full model can be obtained. Therefore, the fine-tuning loss function at this time can be expressed by the following formula (6):

[0136] Loss = Loss_task_specific_head (prediction value, label) + λ*Loss_pretrained_model (prediction value, output of pre-trained model) Formula (6)

[0137] Among them, Loss_task_specific_head is the loss function of the task-specific head network, and Loss_pretrained_model is the loss function of the pre-trained model. You can use a pre-defined loss function:

[0138] L(θ)=-Σy*log(p)+(1-y)*log(1-p) Formula (7)

[0139] Among them, θ is the parameter of the model, y is the label (groundtruth), and p is the probability predicted by the model. λ is the weight coefficient of the two loss functions, which is used to balance the importance of the two. Through back propagation and parameter updates, the model will continuously optimize its weights and biases to better adapt to specific tasks. Therefore, after proper fine-tuning, the large language model can complete the required tasks more accurately.

[0140] Therefore, the embodiment of the present application first obtains a pre-trained basic model based on llama3-8b as a baseline, and uses the instruction set form of the above-mentioned D4, SmileChat, Qiaoban, SoulchatCorpus and other open source data as training data to fine-tune the Lora instructions to obtain the trained psychologist model. In order to verify that the large model after instruction fine-tuning has better empathy and comprehensive ability to ask questions about the Hamilton Depression Rating Scale than the existing models (including but not limited to baseline, etc.), this embodiment can compare the data generation quality of different models by inviting volunteers to manually score, and quantify the comprehensive ability of empathy and Hamilton Depression Rating Scale questions for different models. In the experiment, a total of 10 patients' consultation dialogue data were randomly extracted from the verification data set. For each patient's dialogue data input, the trained psychologist model and other large models were used to generate replies. Finally, 10 volunteers were invited to score the response generation results of different large models during the consultation of these 10 patients, and the scoring criteria are shown in the following table. Through this verification method, the comprehensive ability differences between the psychologist model after instruction fine-tuning training and the baseline model before training and other open source models in terms of dialogue anthropomorphism, empathy and healing, and independent questioning of scale questions were verified. The results can be seen in Tables 1 and 2 below:

[0141]

[0142] Table 1

[0143] The relevant grade descriptions of the scoring may include but are not limited to excellent: 8 points; good: 6 points; average: 4 points; poor: 2 points; very poor: 0 points.

[0144]

[0145] Table 2

[0146] Step 250, obtaining a question-answer data pair set based on the questions asked in the previous round of dialogue and the input speech set.

[0147] The question-answer data pair set includes at least one question-answer data pair.

[0148] For the description of step 250, reference may be made to the above step 130, which will not be described in detail in this embodiment.

[0149] Step 260 , performing sentiment semantic analysis based on the question-answer data pair set to obtain semantic key information, wherein the semantic key information includes a key question-answer data pair set.

[0150] In the specific implementation, during the consultation and communication process between the psychologist big model and the user, data preprocessing is performed based on the user's input voice and caring reply information to achieve emotional semantic analysis, including but not limited to extracting the semantic key information of the user's answer to form a <question-answer> data pair as a key question and answer data pair, and using the key question and answer data pairs to form a key question and answer data pair set. This facilitates the subsequent multimodal quantitative scoring of the psychologist big model. Specifically, in this embodiment, the method of extracting the semantic key information of the user's answer mainly relies on the psychologist big model's understanding and analysis capabilities of semantics. For topic dialogue data with a certain flexible dialogue process, it can accurately extract the semantics of sentences related to the question, which provides an accurate and objective basis for the quantitative scoring and consultation results of the psychologist big model.

[0151] In actual implementation, this embodiment can combine the psychologist's large model quantitative scoring prompt calculation unit and the natural language processing tokenizer to pre-process the consultation dialogue data into serialized and vectorized basic words.

[0152] In an optional embodiment, the embodiment of the present application performs sentiment semantic analysis based on the question-answer data pair set to obtain semantic key information, and the semantic key information includes a key question-answer data pair set, which may specifically include: performing data conversion based on the question-answer data pair set to obtain discrete data; performing sentiment semantic segmentation on the discrete data according to preset segmentation rules to obtain semantic key information.

[0153] In a specific implementation, this embodiment can use a tokenizer segmenter to implement data conversion of text conversation data. The tokenizer segmenter can be used as a key tool in the NLP (natural language processing) preprocessing step. Its core function is to convert the original text data into a format that the model can understand and process. The input conversation data text is further converted into a discrete, structured vector unit data form to obtain discrete data. The text is divided into independent vocabulary units (tokens) in discrete data according to certain rules or algorithms. These vocabulary units can be words, subwords, characters, etc., so that the psychologist model can then calculate and reason about the vectors of these vocabulary units. Taking the BERT model in the HuggingFaceTransformers library as an example, the Tokenizer (such as BertTokenizer) used by it can convert the input text into a format suitable for BERT model input. This process includes dividing the text into tokens, adding special tokens (such as [CLS] and [SEP]) to mark the beginning and end of the sentence, and mapping the tokens to the corresponding ID in the BERT vocabulary. The final output integer sequence will be used as the input data of the BERT model, thus obtaining a set of question-answer data pairs. The tokenizer processing process of the data can be as follows: Figure 8 shown.

[0154] Step 270, using the psychologist model, quantitatively score the set based on the key question and answer data to obtain a multimodal quantitative score.

[0155] In this embodiment, the multimodal quantitative scoring mainly includes quantitative scoring of three dimensions, including but not limited to: quantitative scoring of text dimension (text data score used to characterize topic dialogue), quantitative scoring of image dimension and quantitative scoring of voice dimension. The psychologist model can combine multiple dimensions and perform multimodal quantification based on data question and answer pairs to obtain quantitative scores corresponding to multiple dimensions as multimodal quantitative scoring for subsequent consultation judgment.

[0156] In actual implementation, the embodiments of the present application can use a large model quantization scoring calculation unit (prompt engineering) to perform multimodal quantization scoring, and receive data for quantization scoring through the large model quantization scoring calculation unit (or quantization scoring unit, etc.) to achieve multimodal quantization.

[0157] For example, refer to Fig. 9The workflow diagram of the quantitative scoring unit of the large model shown in Figure 1 is that for the question topic "interest" of the Hamilton Depression Rating Scale, the user's reply voice text is combined with the pre-set "doctor prompt", and the clinical quantitative scoring criteria of the scale are added. The combined text information sequence (as shown in Table 4) is further encoded and processed by the tokenizer into a word vector group that the large model can understand. The large model analyzes the semantic information of <question-answer> through the effective data of the topic dialogue, and finally we can obtain the quantitative scoring output of the topic unit based on the semantic analysis ability of the large model by comparing with the clinical quantitative scoring criteria. After all the 7 questions of the Hamilton Depression Rating Scale are asked, the large model AI psychologist will further review the entire conversation process and give additional scores for the three additional scale evaluation items. Among them, the additional three scale evaluation items include "depression level", "anxiety level", and "self-awareness". The scoring criteria of the above items are more suitable for scoring from the perspective of the doctor's other reviewer. Compared with the existing self-evaluation scale, the self-evaluation from the perspective of a single patient is more objective, thereby more accurately evaluating the patient's depressive psychological state.

[0158] In addition, the ability of the large model to adapt to downstream tasks after applying the prompt in the inference phase is mainly due to its strong pre-training foundation and the guiding role of the prompt. In the pre-training phase, the large model has learned a wealth of language knowledge and common sense and mastered a wide range of contextual information. In the inference phase, through carefully designed or automatically generated prompts, the requirements and contextual information of downstream tasks can be effectively conveyed to the model, guiding it to generate outputs that meet the task requirements. This ability enables the large model to flexibly respond to a variety of complex downstream tasks without the need for complete fine-tuning training for each task.

[0159] For example, Table 3 and Table 4 are combined with Figure 4 The complete process from question-answering to multimodal quantification is shown in Table 3, where Table 3 is a complete example of a Hamilton Depression Rating Scale dialogue, which is a complete dialogue record of a patient consultation process. All HAMD scale topics involved in the dialogue can be found in Figure 3 and Figure 4 Table 4 is an example of a large model quantitative scoring unit.

[0160]

[0161]

[0162] Table 3. Example of a complete Hamilton Depression Rating Scale conversation

[0163]

[0164] Table 4 Example of large model quantitative scoring unit

[0165] In actual processing, multi-dimensional quantitative scoring can more accurately and objectively simulate the process of human psychologists' consultation, thereby giving more objective quantitative scoring results. The quantitative scoring unit receives the processing operation mode of multimodal data. In addition to the semantic quantitative scoring of text data, the scoring items can also receive multimodal comprehensive quantitative scoring of image, voice and other data.

[0166] In an optional embodiment, the embodiment of the present application uses the psychologist model to perform quantitative scoring on the set based on the key question and answer data to obtain a multimodal quantitative score, which may specifically include: when performing caring empathy output on the target object, obtaining the expression video data and audio data of the target object; performing emotional dimension recognition based on the expression video data to obtain image dimension information, and performing emotional dimension recognition on the audio data to obtain voice dimension information; performing topic analysis on the set according to the question and answer data to obtain scoring dimension information; based on the image dimension information, weighted processing is performed in combination with the voice dimension information and the scoring dimension information to obtain a multimodal quantitative score.

[0167] In actual implementation, during the consultation dialogue between the psychologist model and the user, the user's micro-expressions can be collected in real time to obtain expression video data as a scoring item in the image dimension; the user's voice information can be collected to realize the collection of the user's emotions and emotional state, and audio data can be obtained as a scoring item in the voice dimension; the question and answer data set is used as the main scoring dimension for the scoring of topic dialogues. Subsequently, the expression video data can be used to perform micro-expression recognition to obtain image dimension information; the audio data can be used to perform emotional dimension recognition to obtain voice dimension information; the question and answer data set can be used for topic analysis to obtain scoring dimension information, and multiple dimensional information can be combined and weighted according to the preset weights corresponding to the dimensional information, so as to obtain a more objective quantitative scoring result as a multimodal quantitative scoring. It can be seen that the embodiment of the present application can more accurately and objectively simulate the process of human psychologist consultation through multi-dimensional quantitative scoring.

[0168] In actual implementation, the mathematical expression of multimodal quantitative scoring can be expressed by the following formula (8):

[0169] Score=αS_text+βS_image+γS_audio Formula (8)

[0170] Among them, S_text is the main scoring dimension, that is, the text data score of the topic dialogue, and α is the corresponding preset weight; S_image is the scoring item of the image dimension, which can come from the collection and recognition of user micro-expressions during the evaluation process, and β is the corresponding preset weight; S_audio is the scoring item of the voice dimension, which can come from the collection and recognition of user emotions and emotional states during the evaluation process, and γ is the corresponding preset weight. Multimodal data is weighted according to a certain weight configuration to obtain the result of quantitative scoring. In this embodiment, the quantitative scoring method of S_image and S_audio can be implemented by expert annotation, and the expression images and voice data collected during the consultation process are quantitatively annotated in terms of emotions according to certain standards, such as annotating the user's emotions from negative (sadness, anger, anxiety, etc.) to neutral, and then to positive states with 0-10 points, and then training the quantitative scoring model of each dimension through neural network or machine learning technology, and performing multi-dimensional quantitative scoring on the user's multimodal data to obtain more objective and accurate evaluation results.

[0171] It should be noted that, in this embodiment, the values ​​of the weight parameters α, β and γ of the multimodal data can be set as needed, and this embodiment does not limit this. Preferably, the values ​​of the weight parameters α, β and γ can be set to [0.8, 0.1, 0.1] in the initial stage to provide a larger weight configuration for the quantitative score of the text data, and then further data fitting training is performed to determine a more appropriate overall parameter configuration.

[0172] Step 280 , mapping diagnosis is performed according to the multimodal quantitative score to obtain diagnostic information, where the diagnostic information is used to characterize the symptom condition of the target object.

[0173] In this embodiment, the diagnosis information may include the user's medical consultation results, which are used to characterize the user's specific medical condition. Taking depression as an example, the user's medical consultation results may include but are not limited to: normal, possible depression, depression, and severe depression.

[0174] In a specific implementation, this embodiment can preset a mapping table as a basis for mapping between the scores corresponding to the multimodal quantitative information and the results of the consultation. According to the standardized consultation process, the consultation with the user is repeated continuously until all scale questions and supplementary evaluation items are scored, and the corresponding multimodal quantitative scores can be obtained. Subsequently, the consultation results corresponding to the scores can be determined according to the mapping table, thereby reflecting the actual symptoms of the user as diagnostic information. For example, taking the diagnosis of the Hamilton Depression Rating Scale as an example, its mapping table can be shown in Table 5 below:

[0175] Score range Corresponding diagnostic results 0-6 normal 7-17 Possibly depression 18-24 depression >24 Severe depression

[0176] Table 5 Clinical explanation of the Hamilton Depression Rating Scale comprehensive score corresponding to the questionnaire results

[0177] In actual implementation, this embodiment can be provided with a comprehensive diagnosis calculation unit, and a total of 7 rounds of dialogue quantitative scoring and 3 supplementary scoring from other evaluation angles are performed around the user's depression screening using the clinical-level Hamilton Depression Rating Scale, for a total of ten scorings. A professional psychological big model is used to perform word-level semantic analysis and content understanding according to the clinical quantitative scoring criteria of the scale to obtain quantitative scores for conversations on different topics, and the comprehensive diagnosis calculation unit is responsible for accumulating and counting various scores. Finally, the total score of the large model quantitative evaluation provides a mapping basis for the score-questionnaire result based on the clinical description of the final score of the Hamilton Depression Rating Scale (as shown in Table 4). As a result, the mental health screening system of this application completes the entire process of quantitative scoring screening for depression.

[0178] Step 290, performing visualization processing based on the diagnostic information to obtain the target object's consultation result, wherein the consultation result includes a visualized consultation report diagram.

[0179] In a specific implementation, this embodiment can perform visualization processing based on the user's diagnostic information to obtain a visualized consultation report diagram, so that the results of this consultation can be displayed in the form of a visualized report. The visualized report can introduce the results of this consultation / consultation to the patient in a rich form of expression, supplemented by report formats such as bar charts and radar charts, and the visualized consultation report diagram can include not only a detailed consultation process and consultation results, but also: the psychologist model customizes personalized health suggestions for the patient based on the consultation process and diagnosis results.

[0180] Furthermore, in the present application, the system provides a record of the consultation process. Specifically, the clinical quantitative scoring standard for each scoring item, the conversation data of the topic, and the scoring of the psychologist's large model can all be recorded, and a retrospective reference to the consultation process is provided in the form of an appendix pdf material. By recording the quantitative scores during the consultation process, good explainability is provided for the diagnosis of AI quantitative psychological abnormality screening based on the large model. This embodiment can use the open source real-world depression diagnosis conversation data set D4 to verify the various operations performed by the psychologist model during the consultation process and the final consultation results. In the verification data set, a total of 151 conversation data information from real-world patients is included in the process of the large model AI quantitative evaluation test. The consultation and scoring specifications of the process conform to the standardized consultation process of the Hamilton Depression Rating Scale described in this embodiment. Among the 151 patients, 22 (14.57%) were labeled as "normal", 59 (39.07%) were labeled as "possibly depressed", 61 (40.40%) were labeled as "definitely depressed", and 9 (5.96%) were labeled as "severely depressed". The distribution of the number of depressed patients in this validation dataset is shown in Table 6.

[0181] By combining the large model with a quantitative evaluation method of a clinical-level medical depression scale, this embodiment realizes a technical method for rapid screening of mental illness in the population with a large model standardization, and achieves very good results in the validation data set. In the end, the standardized large model quantitative evaluation results of 151 real-world patients achieved a two-category disease diagnosis rate of 97.35%, and a four-category diagnosis accuracy rate of 80.79%. Other statistically significant indicators are shown in Table 7 below.

[0182]

[0183] Table 6 Distribution of depression diagnostic labels in the validation dataset

[0184]

[0185] Table 7 Depression quantitative evaluation diagnosis results

[0186] In summary, the embodiment of the present application continuously obtains the voice information of the target object during the consultation interaction process with the target object, so as to perform caring empathy output based on the requirement of caring reply for each voice information in the input voice set through the psychologist model, and then extracts the other-rated mental scale from the preset scale database, and extracts evaluation-related scale questions from the other-rated mental scale for the voice information, and obtains a question and answer data pair set based on the questions asked in the previous round of dialogue and the input voice set, and performs emotional semantic analysis based on the question and answer data pair set to obtain semantic key information, and performs quantitative scoring based on the key question and answer data pair set contained in the semantic key information through the psychologist model to obtain a multimodal quantitative score, and then performs mapping diagnosis based on the multimodal quantitative score to obtain diagnostic information, so as to characterize the condition of the target object through the diagnostic information, and then performs visualization processing based on the diagnostic information to obtain the consultation result of the target object including a visualized consultation report diagram. Therefore, the embodiment of the present application provides a specific implementation route of a large language model AI quantitative evaluation method for depression and other psychological abnormalities and its technology. By analyzing the scale diagnosis dialogue data of patients in the real world, key semantic features and emotional indicators can be extracted, thereby providing an objective, accurate and efficient method for auxiliary rapid diagnosis and screening of depression. Finally, combined with digital human interaction technology, a complete, accurate and efficient diagnosis system for depression and other psychological diseases is established, which can realize standardized AI quantitative evaluation of psychological abnormalities to assist doctors in early diagnosis and intervention of psychological diseases such as depression. It can be seen that compared with the ambiguity and lack of specificity of existing AI quantitative evaluation methods, the embodiment of the present application has higher accuracy and better actual application effect in practical applications, high diagnostic accuracy, and strong interpretability. It can truly become a clinically recognized analysis and diagnosis method, and effectively solves various problems existing in the prior art.

[0187] It should be noted that, for the purpose of simple description, the method embodiments are expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited to the described order of actions, because according to the embodiments of the present application, certain steps may be performed in other orders or simultaneously.

[0188] like Fig.10 As shown, the embodiment of the present application also provides a quantitative diagnosis device 1000 for abnormal psychological diseases, including:

[0189] An acquisition module 1010 is used to continuously acquire an input speech set of the target object during the consultation interaction process with the target object, wherein the input speech set includes at least one piece of speech information;

[0190] The empathy output module 1020 is used to perform caring empathy output based on the requirements of caring response through the psychologist model for each voice message in the input voice set, and extract evaluation-related scale questions from a preset other-assessment mental scale through the preset psychologist model;

[0191] A question-answer data pair set determination module 1030, configured to obtain a question-answer data pair set according to the question asked in the previous round of dialogue and the input speech set, wherein the question-answer data pair set includes at least one question-answer data pair;

[0192] The multimodal quantitative processing module 1040 is used to perform multimodal quantitative scoring processing on the set based on the question and answer data, and obtain the diagnosis results of the target object after multiple rounds of cumulative scoring.

[0193] Optionally, the empathy output module 1020 includes:

[0194] A text conversion submodule, used for performing text conversion based on the voice information to obtain voice text;

[0195] An empathic response voice determination submodule, used to determine an empathic response voice based on the voice text through the psychologist model according to the requirements of the caring response;

[0196] A digital human video construction submodule, used to construct a digital human video for the empathy reply voice;

[0197] An empathy output submodule, used to output the empathy reply voice through the digital human video in real-time communication with the target object, so as to perform caring empathy output to the target object;

[0198] An extraction submodule is used to extract the other-rated mental scale from a preset scale database;

[0199] The scale question extraction submodule is used to extract assessment-related scale questions from the other-rated mental scale based on the voice information.

[0200] Optionally, the multi-modal quantization processing module 1040 includes:

[0201] The semantic analysis submodule is used to perform sentiment semantic analysis based on the question-answer data pair set to obtain semantic key information, which includes the key question-answer data pair set.

[0202] A multimodal quantification submodule, for performing quantitative scoring on the set according to the key question and answer data through the psychologist model to obtain a multimodal quantification score;

[0203] A mapping diagnosis submodule, used for performing mapping diagnosis according to the multimodal quantitative score to obtain diagnostic information, wherein the diagnostic information is used to characterize the symptom condition of the target object;

[0204] The visualization processing submodule is used to perform visualization processing based on the diagnostic information to obtain the consultation result of the target object, and the consultation result includes a visualized consultation report diagram.

[0205] Optionally, the semantic analysis submodule includes:

[0206] A data conversion submodule, used for performing data conversion based on the question-answer data set to obtain discrete data;

[0207] The semantic segmentation submodule is used to perform sentiment semantic segmentation on the discrete data according to preset segmentation rules to obtain semantic key information.

[0208] Optionally, the multimodal quantization submodule includes:

[0209] An acquisition unit, configured to acquire expression video data and audio data of the target object when performing caring empathy output to the target object;

[0210] An emotion dimension recognition unit, used to perform emotion dimension recognition based on the expression video data to obtain image dimension information, and perform emotion dimension recognition on the audio data to obtain voice dimension information;

[0211] A topic analysis unit, used to perform topic analysis on the set according to the question and answer data to obtain scoring dimension information;

[0212] A weighted processing unit is used to perform weighted processing based on the image dimension information in combination with the speech dimension information and the scoring dimension information to obtain a multimodal quantitative score.

[0213] Optionally, the quantitative diagnosis device 1000 for abnormal psychological diseases further includes:

[0214] A sample data acquisition module, used to acquire sample question-answer data pairs and construct a sample data set based on the sample question-answer data pairs;

[0215] A first model building module, used to build an initial model according to a preset model building component, wherein the model building component includes a multi-head attention mechanism, a relative position encoding, and a feedforward neural network;

[0216] A model training module, used to perform model training on the initial model based on the sample data set to obtain an initial large language model;

[0217] A loss parameter adjustment module, used for performing loss parameter adjustment processing according to the initial large language model to obtain a target large language model;

[0218] The second model building module is used to build a model based on the target large language model in combination with a preset expert knowledge base to obtain the psychologist model.

[0219] Optionally, the loss parameter adjustment module includes:

[0220] An additional network layer construction submodule, used to determine a specific loss function and a fine-tuning loss function of the initial large language model, and to construct an additional network layer in the initial large language model based on the fine-tuning loss function;

[0221] A first calculation submodule, used to calculate the gradient of the specific loss function with respect to the initial large language model through a preset back propagation algorithm to obtain a first update parameter;

[0222] A second calculation submodule is used to calculate the gradient of the fine-tuning loss function with respect to the additional network layer based on the back propagation algorithm and the specific loss function to obtain a second update parameter;

[0223] A parameter fine-tuning submodule is used to update the parameters of the initial large language model according to the first update parameter, and to fine-tune the parameters of the additional network layer according to the second update parameter to obtain the target large language model.

[0224] It should be noted that the quantitative diagnosis device for abnormal psychological diseases provided in the embodiments of the present application can execute the quantitative diagnosis method for abnormal psychological diseases provided in any embodiment of the present application, and has the corresponding functions and beneficial effects of the execution method.

[0225] In a specific implementation, the quantitative diagnostic device for abnormal mental illness can be integrated into a device, so that the device can make an empathic response and diagnose abnormal mental illness for the target object based on the target object's voice information during the consultation interaction with the target object, and as an electronic device, achieve objective, accurate and efficient rapid diagnosis and screening of auxiliary depression. The electronic device can be composed of two or more physical entities, or it can be composed of one physical entity, such as a personal computer (PC), a computer, a server, etc., and the embodiment of the present application does not impose specific restrictions on this.

[0226] like Fig.11As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114; the memory 113 is used to store computer programs; the processor 111 is used to implement the steps of the quantitative diagnosis method of abnormal psychological diseases provided by any of the aforementioned method embodiments when executing the program stored in the memory 113. Exemplarily, the steps of the quantitative diagnosis method for abnormal psychological diseases may include the following steps: during the consultation interaction process with the target object, continuously obtaining the input voice set of the target object, the input voice set containing at least one voice message; for each voice message in the input voice set, performing caring empathy output based on the requirement of caring response through a psychologist model, and extracting assessment-related scale questions from a preset other-assessment mental scale through a preset psychologist model; obtaining a question-answer data pair set based on the questions asked in the previous round of dialogue and the input voice set, the question-answer data pair set containing at least one question-answer data pair; performing multimodal quantitative scoring processing based on the question-answer data pair set to obtain the consultation result of the target object after multiple rounds of cumulative scoring.

[0227] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the quantitative diagnosis method for abnormal psychological diseases provided in any of the aforementioned method embodiments are implemented.

[0228] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0229] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A quantitative diagnostic method for abnormal psychological diseases, characterized in that: include: During the consultation interaction process with the target object, continuously obtaining an input speech set of the target object, wherein the input speech set includes at least one piece of speech information; For each voice message in the input voice set, a caring empathy output is performed based on the requirement of a caring reply through the psychologist model, and evaluation-related scale questions are extracted from a preset other-assessment mental scale through the preset psychologist model; Obtaining a question-answer data pair set according to the question asked in the previous round of dialogue and the input speech set, wherein the question-answer data pair set includes at least one question-answer data pair; Based on the question-and-answer data, the set is processed with multimodal quantitative scoring to obtain the diagnosis results of the target object after multiple rounds of cumulative scoring.

2. The method according to claim 1, characterized in that For each voice message in the input voice set, the psychologist model performs caring empathy output based on the requirement of caring response, including: Perform text conversion based on the voice information to obtain voice text; According to the requirements of caring response, the psychologist model determines the empathic response voice based on the voice text; Constructing a digital human video for the empathic response voice; In the real-time communication with the target object, the empathic response voice is output through the digital human video to provide caring empathic output to the target object.

3. The method according to claim 1, characterized in that The method of extracting assessment-related scale questions from a preset other-assessment mental scale through a preset psychiatrist model includes: Extract the other-rated mental scale from the preset scale database; With respect to the voice information, assessment-related scale questions are extracted from the other-rated mental scale.

4. The method according to claim 1, characterized in that: The multimodal quantitative scoring process is performed on the set based on the question-answer data to obtain the target object's diagnosis results after multiple rounds of cumulative scoring, including: Performing sentiment semantic analysis based on the question-answer data pair set to obtain semantic key information, wherein the semantic key information includes a key question-answer data pair set; Using the psychologist model, the set is quantitatively scored according to the key question and answer data to obtain a multimodal quantitative score; Performing mapping diagnosis according to the multimodal quantitative score to obtain diagnostic information, wherein the diagnostic information is used to characterize the symptom condition of the target object; Visualization processing is performed based on the diagnostic information to obtain a consultation result of the target object, wherein the consultation result includes a visualized consultation report diagram.

5. The method according to claim 4, characterized in that The psychologist model is used to quantitatively score the set according to the key question and answer data to obtain a multimodal quantitative score, including: When performing caring empathy output to the target object, acquiring expression video data and audio data of the target object; Performing emotion dimension recognition based on the expression video data to obtain image dimension information, and performing emotion dimension recognition on the audio data to obtain voice dimension information; Performing topic analysis on the set according to the question-answer data to obtain scoring dimension information; Based on the image dimension information, weighted processing is performed in combination with the speech dimension information and the scoring dimension information to obtain a multimodal quantitative score.

6. The method according to claim 1, characterized in that The emotional semantic analysis is performed on the set based on the question-answer data to obtain semantic key information, including: Performing data conversion on the set of question-answer data to obtain discrete data; According to the preset segmentation rules, the discrete data is subjected to sentiment semantic segmentation to obtain semantic key information.

7. The method according to any one of claims 1 to 6, characterized in that: Before extracting the caring reply information corresponding to the voice information from the preset evaluation table through the preset psychologist model, the method further includes: Obtaining sample question-answer data pairs, and constructing a sample data set based on the sample question-answer data pairs; Building an initial model according to a preset model building component, wherein the model building component includes a multi-head attention mechanism, a relative position encoding, and a feedforward neural network; Performing model training on the initial model based on the sample data set to obtain an initial large language model; Performing loss parameter adjustment processing according to the initial large language model to obtain a target large language model; The target large language model is used in combination with a preset expert knowledge base to construct a model to obtain the psychologist model.

8. The method according to claim 7, characterized in that The performing loss parameter adjustment processing according to the initial large language model to obtain a target large language model includes: Determining a specific loss function and a fine-tuning loss function of the initial large language model, and constructing an additional network layer in the initial large language model based on the fine-tuning loss function; Calculate the gradient of the specific loss function to the initial large language model through a preset back-propagation algorithm to obtain a first update parameter; Based on the back propagation algorithm and the specific loss function, calculating the gradient of the fine-tuning loss function with respect to the additional network layer to obtain a second update parameter; The initial large language model is updated with parameters according to the first update parameters, and the additional network layer is fine-tuned with parameters according to the second update parameters to obtain the target large language model.

9. A quantitative diagnostic device for abnormal psychological diseases, characterized in that: include: An acquisition module, used for continuously acquiring an input speech set of the target object during the consultation interaction process with the target object, wherein the input speech set includes at least one piece of speech information; an empathy output module, for performing caring empathy output based on the requirement of caring response through a psychologist model for each voice message in the input voice set, and extracting evaluation-related scale questions from a preset other-assessment mental scale through a preset psychologist model; A question-answer data pair set determination module, configured to obtain a question-answer data pair set according to the questions asked in the previous round of dialogue and the input speech set, wherein the question-answer data pair set includes at least one question-answer data pair; The multimodal quantitative processing module is used to perform multimodal quantitative scoring processing on the set based on the question and answer data, and obtain the diagnosis results of the target object after multiple rounds of cumulative scoring.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the quantitative diagnosis method of abnormal psychological diseases as described in any one of claims 1 to 8 when executing the program stored in the memory.

Images