Interactive psychological assessment method, device and equipment and storage medium

Abstract

The invention relates to the field of psychological health assessment, and discloses an interactive psychological assessment method, device and equipment and a storage medium. The method comprises the following steps: converting a question text in a DSM-5 psychological assessment question bank into a corresponding question audio through a text-to-speech model; driving the virtual character image according to the question audio and the question text, and transmitting the question audio to the user; obtaining an answer audio fed back by the user according to the question audio, and converting the answer audio into a corresponding answer text through a voice recognition model; obtaining a face image when the user answers, and obtaining an emotional state of the user through a multi-modal model according to the face image and the answer audio; according to the emotional state and the answer text, a model is generated through voice, and the virtual character image is driven to conduct co-situation reply. In the embodiment of the invention, the interactivity of evaluation can be increased, and the user can be effectively guided to express real emotions, so that the effectiveness and the accuracy of an evaluation result are improved.

Description

Technical Field

[0001] The present invention relates to the field of mental health assessment, and in particular to an interactive psychological assessment method, device, equipment and storage medium. Background Art

[0002] Mental health refers to a continuous and positively developing psychological state in which the subject can adapt well and fully develop his or her physical and mental potential. How to maintain people's mental health and actively intervene and guide those with psychological problems is of great importance to the entire family, society and company.

[0003] Current mental health assessments mainly rely on standardized questionnaires and scale assessments, such as DSM-5-based psychological assessments. Although these have been widely used in clinical work, their results are often rigid, lacking personalization and emotional interaction, and resulting in poor user experience, which in turn affects the effectiveness and accuracy of the assessment. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem of how to improve the effectiveness and accuracy of psychological assessment results.

[0005] A first aspect of the present invention provides an interactive psychological assessment method, the interactive psychological assessment method comprising:

[0006] Through the text-to-speech model, the question text in the DSM-5 psychological assessment question bank is converted into the corresponding question audio;

[0007] According to the question audio and the question text, the virtual character image is driven to transmit the question audio to the user;

[0008] Obtaining the user's answer audio based on the audio feedback of the question, and converting the answer audio into corresponding answer text through a speech recognition model;

[0009] Acquire a facial image of the user when answering, and obtain the emotional state of the user through a multimodal model based on the facial image and the answer audio;

[0010] According to the emotional state and the answer text, the virtual character image is driven to make an empathetic response through a speech generation model.

[0011] Optionally, in a first implementation of the first aspect of the present invention, driving the virtual character image to make an empathic reply according to the emotional state and the answer text includes:

[0012] Converting the answer text into corresponding question options;

[0013] generating an empathic response based on the question options and the emotional state;

[0014] According to the empathy response, the virtual character image is driven by a speech generation model to output the empathy response in speech;

[0015] The question options and the emotional state are summarized to generate a psychological assessment report.

[0016] Optionally, in a second implementation of the first aspect of the present invention, the step of driving the virtual character image to voice-output the empathy response through a voice generation model according to the empathy response includes:

[0017] Choose the corresponding tone according to the dialogue scene;

[0018] According to the empathic response, an empathic response audio is obtained through a speech generation model;

[0019] According to the timbre and the empathic response audio, the virtual character image is driven by timbre fusion technology to voice output the empathic response.

[0020] Optionally, in a third implementation of the first aspect of the present invention, converting the answer text into corresponding question options includes:

[0021] Extracting keywords from the answer text and disambiguating the keywords by natural language processing technology;

[0022] According to the disambiguated keywords, the question options corresponding to the keywords are matched by using the approximate word calculation technology.

[0023] Optionally, in a fourth implementation of the first aspect of the present invention, obtaining the answer audio of the user based on the audio feedback of the question, and converting the answer audio into a corresponding answer text through a speech recognition model includes:

[0024] Obtaining the user's answer audio based on the audio feedback of the question;

[0025] Determining, by a speech recognition model, whether the answer audio is in a preset language;

[0026] If not, translating the answer audio into the corresponding preset language through the speech recognition model;

[0027] The translated answer audio is converted into corresponding answer text through the speech recognition model.

[0028] Optionally, in a fifth implementation of the first aspect of the present invention, after converting the translated answer audio into a corresponding answer text by the speech recognition model, the method further includes:

[0029] Obtaining the real-time emotional state of the user;

[0030] Dynamically adjusting the question settings in the DSM-5 psychological assessment question bank according to the real-time emotional state;

[0031] The adjusted question is output to the user by driving the virtual character image voice.

[0032] Optionally, in a sixth implementation of the first aspect of the present invention, the step of driving the virtual character image according to the question audio and the question text to transmit the question audio to the user includes:

[0033] The question audio and the question text are combined through a deep learning algorithm to generate an expression coefficient of the virtual character image;

[0034] According to the expression coefficient, combined with 3D model binding and facial animation technology, the lip shape and facial expression of the virtual character are generated to drive the virtual character and transmit the question audio to the user.

[0035] A second aspect of the present invention provides an interactive psychological assessment device, comprising: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the instructions in the memory so that the interactive psychological assessment device performs the above-mentioned interactive psychological assessment method.

[0036] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned interactive psychological assessment method.

[0037] In an embodiment of the present invention, the question text in the DSM-5 psychological assessment question bank is converted into corresponding question audio through a text-to-speech model; based on the question audio and the question text, a virtual character image is driven to transmit the question audio to the user; the answer audio of the user based on the question audio feedback is obtained, and the answer audio is converted into a corresponding answer text through a speech recognition model; the facial image of the user when answering is obtained, and the emotional state of the user is obtained through a multimodal model based on the facial image and the answer audio; based on the emotional state and the answer text, the virtual character image is driven to perform an empathic response through a speech generation model. In the present invention, through the virtual character image, not only the interactivity of the assessment is increased, but also the user can be effectively guided to express real emotions, thereby improving the effectiveness and accuracy of the assessment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of an embodiment of an interactive psychological assessment method in an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of an embodiment of an interactive psychological assessment device in an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of an embodiment of an interactive psychological assessment device in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The embodiments of the present invention provide an interactive psychological assessment method, apparatus, device and storage medium.

[0042] The embodiments disclosed in the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments disclosed in the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments disclosed in the present invention are only for exemplary purposes and are not intended to limit the scope of protection disclosed in the present invention.

[0043] In the description of the embodiments disclosed in the present invention, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0044] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the interactive psychological assessment method in the embodiment of the present invention includes:

[0045] S100, through a text-to-speech model, converts the question text in the DSM-5 psychological assessment question bank into the corresponding question audio.

[0046] In this embodiment, the text-to-speech model (Text-to-Speech, TTS) is a technology for converting text into natural speech. Specifically, the steps of converting the question text in the DSM-5 psychological assessment question bank into the corresponding question audio include: first, the system needs to analyze the question text in the DSM-5 psychological assessment question bank, including sentence segmentation, word segmentation, part of speech (POS), etc. This step is similar to the preprocessing stage in natural language processing, and the purpose is to convert the question text into a more easily processed form. After the question text is analyzed, the text will be converted into a phoneme sequence. The phoneme is the basic unit of speech and the smallest sound segment with speech meaning in pronunciation. The purpose of this step is to decompose each word in the text into its constituent phonemes. Then, the phoneme sequence will be converted into acoustic features, such as pitch, volume and intonation, etc. These acoustic features are key parameters for generating speech. Acoustic models are usually based on different technologies such as statistical parameters, waveform splicing, hybrid methods or end-to-end neural networks. Finally, the acoustic features are used to generate the waveform of the speech. This step usually involves using a vocoder to convert the acoustic features into an audible speech signal.

[0047] S200, driving the virtual character image according to the question audio and question text, and transmitting the question audio to the user.

[0048] In this embodiment, a virtual character image is designed, which can use natural language processing technology to achieve a more natural conversation. After acquiring the user's voice content, the virtual character driving technology can be used to achieve the virtual character's mouth shape change and expression synchronization. In addition, a motion capture system can also be used to drive the facial and hand movements of the virtual digital person. In order to transmit the problem audio to the user, a variety of audio transmission technologies can be used. For example, Dante digital audio transmission technology is based on IP network and provides low-latency, high-precision audio transmission solutions.

[0049] S300, obtaining the answer audio of the user based on the audio feedback of the question, and converting the answer audio into corresponding answer text through a speech recognition model.

[0050] In this embodiment, the answer audio of the user feedback is converted into the corresponding answer text through the speech recognition model, wherein the speech recognition model can be a Whisper model, and the Whisper model adopts the standard Transformer encoder-decoder architecture, which means that it processes the input data through the encoder-decoder structure, and this architecture has been widely used in speech recognition tasks. Specifically, first, the input answer audio will be divided into 30-second segments and converted into a log-Mel spectrogram. This step includes sampling, window function application, fast Fourier transform (FFT), Mel filter bank application, and finally obtaining the logarithmic Mel energy. The obtained log-Mel spectrogram will be provided to the Whisper model as an input feature. Then, the input log-Mel spectrogram is passed to the encoder for processing. The role of the encoder is to extract key features in the audio signal, which are then used to generate text output. Finally, the answer text is generated by the decoder.

[0051] S400, obtaining a facial image of the user when answering, and obtaining the user's emotional state through a multimodal model based on the facial image and the answer audio.

[0052] In this embodiment, facial expression features and voice intonation features are extracted from the facial image and the answer audio, and the facial expression features and voice intonation features are integrated through a multimodal model to obtain the user's emotional state. Optionally, before extracting the voice intonation features, denoising can be performed. Denoising is a key preprocessing step. The noise may come from background noise, electronic noise of the device, and interference from the microphone. By removing these noises, the clarity of the voice signal can be significantly improved, thereby improving the accuracy of subsequent voice intonation feature extraction. Before extracting the facial expression features, the image data can also be optimized, including steps such as illumination change normalization and noise reduction, to improve the detection accuracy. Illumination normalization can reduce the impact of illumination changes on the recognition results, while noise reduction can improve image quality and reduce noise interference. In addition, the brightness of the facial image can be adjusted to expand the number of samples and weaken the impact of illumination problems on expression recognition.

[0053] S500, based on the emotional state and the answer text, drives the virtual character image to make an empathetic response through the voice generation model.

[0054] In this embodiment, the virtual character image can interact with the user through voice, thereby enhancing the user's willingness to interact with audio and video, thereby obtaining the user's audio and video data for the smooth execution of AI empathy technology. This technology greatly improves the interactivity of the evaluation process by simulating real dialogue scenarios. The virtual character image can adjust the interaction method in time according to the user's tone and emotional changes. Specifically, if the user shows a large emotional fluctuation (obtained using multimodal real-time emotion recognition analysis), the system will take the initiative to care and give the user a suitable empathetic response, allowing the user to express himself in a comfortable environment, thereby improving the authenticity and accuracy of the evaluation results.

[0055] In an optional implementation of the first aspect of the present invention, driving the virtual character image to perform empathic response according to the emotional state and the answer text includes:

[0056] Convert the answer text into corresponding question options; generate empathetic responses based on the question options and emotional states; based on the empathetic responses, drive the virtual character image through the voice generation model to voice output the empathetic responses; summarize the question options and emotional states to generate a psychological assessment report.

[0057] In this embodiment, a more efficient recognition mechanism is introduced into the speech recognition model, for example, the user's answer text can be converted into question options through rule matching, which greatly increases the flexibility of use. Then, according to the question options and emotional state, an empathic response is generated through a large language model. Specifically, first, the question options and emotional state are input into the large language model. After the model understands the input content, the model needs to generate appropriate empathic responses, which usually involves generating supportive messages. These messages should be related to the user's life experience and can cause emotional resonance. For example, let the model make empathic responses to posts describing daily life experiences, such as workplace situations, parenting, interpersonal relationships, etc. In order to enhance the empathy effect, a multimodal response generation framework such as PEGS can be used. This framework not only generates text replies, but also combines multimedia elements such as emoticons to convey emotions more intuitively. After generating the empathic response text, the empathic response text is converted into speech through a speech generation model. The generated speech needs to drive the virtual character image for speech output, which is usually achieved through speech recognition and animation generation technology.

[0058] In an optional implementation of the first aspect of the present invention, according to the empathy response, driving the virtual character image through the speech generation model to voice output the empathy response includes:

[0059] According to the dialogue scenario, the corresponding timbre is selected; according to the empathetic response, the empathetic response audio is obtained through the voice generation model; according to the timbre and the empathetic response audio, the virtual character image is driven by the timbre fusion technology to output the empathetic response through voice.

[0060] In this embodiment, the user can select a suitable timbre according to the needs of the dialogue scene to ensure that the voice output is more in line with the scene requirements. Optionally, the ChatTTS model can be used, which is designed for dialogue scenes and provides a variety of timbre options, including male and female voices. The ChatTTS model can predict and control fine rhythmic features, including laughter, pauses, and interjections, which allows users to adjust the tone, timbre, and speed of speech by inserting control symbols. In addition, users can also make more detailed timbre adjustments through the WebUI interface. Then, the empathy response text is converted into speech through the speech generation model, and then the empathy response audio is fused with the selected timbre through the timbre fusion technology to obtain the empathy response audio with the target timbre. The empathy response audio with the target timbre is combined with the virtual character image using the virtual image voice animation synthesis technology to drive the virtual character image to output the empathy response through speech.

[0061] In an optional implementation of the first aspect of the present invention, converting the answer text into corresponding question options includes:

[0062] Through natural language processing technology, keywords are extracted from the answer text and disambiguated; based on the disambiguated keywords, approximate word calculation technology is used to match the question options corresponding to the keywords.

[0063] In this embodiment, first, the text summarization technology in the natural language processing technology is used to extract the key information of the answer text, and then, the important words and phrases are extracted from the text through the keyword extraction technology. After extracting the keywords, word sense disambiguation is required to ensure that the computer correctly understands the specific meaning of each word in the context, wherein the disambiguation method includes context-based disambiguation, dictionary-based disambiguation, corpus-based disambiguation, etc. For example, dictionary resources such as WordNet can be used to assist word sense disambiguation, or supervised learning, unsupervised learning and other methods can be used to determine the correct meaning of the keywords. Finally, the approximate word calculation technology is used to match the question options corresponding to the keywords, wherein the approximate word calculation can be based on word vectors, synonyms, etc., and the similarity between words is calculated to find the most matching option.

[0064] In an optional implementation of the first aspect of the present invention, obtaining the answer audio of the user according to the audio feedback of the question, and converting the answer audio into the corresponding answer text through the speech recognition model includes:

[0065] Through the speech recognition model, determine whether the answer audio uses the preset language; if not, translate the answer audio into the corresponding preset language through the speech recognition model; and convert the translated answer audio into the corresponding answer text through the speech recognition model.

[0066] In this embodiment, the speech recognition model is a multi-task model that can perform multilingual speech recognition, speech translation and language recognition tasks, which means that it can not only recognize and transcribe speech, but also translate speech into other languages ​​or recognize the language used; in order to achieve multi-tasking, the speech recognition model uses a set of special tags to specify different tasks, such as language recognition, speech translation, etc. These special tags serve as task specifiers or classification targets, so that the model can jointly represent all tasks in a single decoder predicted token sequence. Specifically, first obtain the answer audio of the user based on the audio feedback of the question, identify the language used in the answer audio through the speech recognition model, and determine whether the answer audio uses the preset language. If not, the answer audio is translated into the corresponding preset language through the speech recognition model. Finally, the translated answer audio is converted into the corresponding answer text through the speech recognition model.

[0067] In an optional implementation of the first aspect of the present invention, after converting the translated answer audio into the corresponding answer text through the speech recognition model, the method further includes:

[0068] Obtain the user's real-time emotional state; dynamically adjust the question settings in the DSM-5 psychological assessment question bank based on the real-time emotional state; output the adjusted questions to the user by driving the virtual character's voice.

[0069] In this embodiment, the real-time emotional state of the user's feedback is obtained, and the evaluation process and question settings are dynamically adjusted according to the user's real-time emotional state feedback, so that the evaluation process is more personalized and humane, thereby enhancing the user's sense of participation and the accuracy of the evaluation, and the adjusted questions are output to the user by driving the virtual character voice.

[0070] In an optional implementation of the first aspect of the present invention, driving the virtual character image according to the question audio and the question text to transmit the question audio to the user includes:

[0071] Through deep learning algorithms, the question audio and question text are combined to generate the expression coefficient of the virtual character. Based on the expression coefficient, combined with 3D model binding and facial animation technology, the lip shape and facial expression of the virtual character are generated to drive the virtual character and transmit the question audio to the user.

[0072] In this embodiment, given the question audio, the expression of the mouth shape will be predicted. This step usually uses spectral features to represent the audio, which is used to guide the mouth shape changes of the virtual character image. Further, through the deep learning algorithm, the question audio features and the question text information are combined to generate the expression coefficients of the corresponding virtual character image. These expression coefficients are used to drive the mouth shape and facial expressions of the virtual character image. Then, using the generated expression coefficients, combined with 3D model binding and facial animation technology, the mouth shape and facial expressions of the virtual character image are generated to ensure that the mouth shape of the virtual character image perfectly matches the input voice content. Specifically, to generate realistic facial expressions, it is necessary to bind the 3D model and add bones and controllers to the built model torso and facial expressions. Then, facial animation technology is used to generate facial animation. The target virtual face feature points can be generated by using a generative adversarial network, and the two-dimensional facial expression of the current posture can be rendered in real time. A muscle model based on muscle fibers can also be used to control facial expressions using a nonlinear hybrid skin method, thereby achieving realistic facial animation. Secondly, audio-driven facial animation can generate facial animation of a virtual character based on the input speech sequence, recreating the scene of a person speaking.

[0073] See also Figure 2 According to a second aspect of the present invention, there is provided an interactive psychological assessment device, the interactive psychological assessment device comprising:

[0074] The question text conversion module 10 is used to convert the question text in the DSM-5 psychological assessment question bank into the corresponding question audio through a text-to-speech model;

[0075] The image driving module 20 is used to drive the virtual character image according to the question audio and question text, and transmit the question audio to the user;

[0076] The answer audio conversion module 30 is used to obtain the answer audio of the user based on the question audio feedback, and convert the answer audio into the corresponding answer text through the speech recognition model;

[0077] The emotional state acquisition module 40 is used to acquire the facial image of the user when answering, and obtain the emotional state of the user through a multimodal model based on the facial image and the answer audio;

[0078] The empathy response module 50 is used to drive the virtual character image to perform empathy response based on the emotional state and the answer text through the voice generation model.

[0079] In an optional implementation of the second aspect of the present invention, the empathy response module 50 is also used to convert the answer text into corresponding question options; generate empathy responses based on the question options and emotional states; based on the empathy responses, drive the virtual character image through a voice generation model to voice output the empathy responses; summarize the question options and emotional states to generate a psychological assessment report.

[0080] In an optional implementation of the second aspect of the present invention, the empathy response module 50 is further configured to select a corresponding timbre according to the dialogue scene;

[0081] Based on the empathetic response, the empathetic response audio is obtained through the voice generation model; based on the timbre and the empathetic response audio, the virtual character image is driven by the timbre fusion technology to output the empathetic response through voice.

[0082] In an optional implementation of the second aspect of the present invention, the empathy response module 50 is also used to extract keywords from the answer text through natural language processing technology and disambiguate the keywords; based on the disambiguated keywords, the question options corresponding to the keywords are matched through approximate word calculation technology.

[0083] In an optional implementation of the second aspect of the present invention, the answer audio conversion module 30 is also used to obtain the answer audio of the user based on the question audio feedback; through the speech recognition model, determine whether the answer audio uses a preset language; if not, translate the answer audio into the corresponding preset language through the speech recognition model; and convert the translated answer audio into the corresponding answer text through the speech recognition model.

[0084] In an optional implementation of the second aspect of the present invention, the interactive psychological assessment device further includes:

[0085] The dynamic adjustment module obtains the user's real-time emotional state; dynamically adjusts the question settings in the DSM-5 psychological assessment question bank according to the real-time emotional state; and outputs the adjusted questions to the user by driving the virtual character voice.

[0086] In an optional implementation of the second aspect of the present invention, the image driving module 20 is also used to combine the question audio and the question text through a deep learning algorithm to generate an expression coefficient of the virtual character image; based on the expression coefficient, combined with 3D model binding and facial animation technology, the lip shape and facial expression of the virtual character image are generated to drive the virtual character image and transmit the question audio to the user.

[0087] Figure 3is a schematic diagram of the structure of an interactive psychological assessment device provided by an embodiment of the present invention. The interactive psychological assessment device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 may be temporary storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the interactive psychological assessment device 500. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the interactive psychological assessment device 500.

[0088] The interactive psychological assessment device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3 The structure of the interactive psychological assessment device shown does not constitute a limitation on the interactive psychological assessment device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0089] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the interactive psychological assessment method.

[0090] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0092] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An interactive psychological assessment method, characterized in that: The interactive psychological assessment method includes: Through the text-to-speech model, the question text in the DSM-5 psychological assessment question bank is converted into the corresponding question audio; According to the question audio and the question text, the virtual character image is driven to transmit the question audio to the user; Obtaining the user's answer audio based on the audio feedback of the question, and converting the answer audio into corresponding answer text through a speech recognition model; Acquire a facial image of the user when answering, and obtain the emotional state of the user through a multimodal model based on the facial image and the answer audio; According to the emotional state and the answer text, the virtual character image is driven to make an empathetic response through a speech generation model.

2. The interactive psychological assessment method according to claim 1, characterized in that: The step of driving the virtual character image to perform empathic response according to the emotional state and the answer text includes: Converting the answer text into corresponding question options; generating an empathic response based on the question options and the emotional state; According to the empathy response, the virtual character image is driven by a speech generation model to output the empathy response in speech; The question options and the emotional state are summarized to generate a psychological assessment report.

3. The interactive psychological assessment method according to claim 2, characterized in that: The step of driving the virtual character image to output the empathy response by voice using a voice generation model according to the empathy response includes: Choose the corresponding tone according to the dialogue scene; According to the empathic response, an empathic response audio is obtained through a speech generation model; According to the timbre and the empathic response audio, the virtual character image is driven by timbre fusion technology to voice output the empathic response.

4. The interactive psychological assessment method according to claim 2, characterized in that: The step of converting the answer text into corresponding question options includes: Extracting keywords from the answer text and disambiguating the keywords by natural language processing technology; According to the disambiguated keywords, the question options corresponding to the keywords are matched by using the approximate word calculation technology.

5. The interactive psychological assessment method according to claim 1, characterized in that: The step of obtaining the answer audio of the user according to the audio feedback of the question, and converting the answer audio into corresponding answer text through a speech recognition model comprises: Obtaining the user's answer audio based on the audio feedback of the question; Determining, by a speech recognition model, whether the answer audio is in a preset language; If not, translating the answer audio into the corresponding preset language through the speech recognition model; The translated answer audio is converted into corresponding answer text through the speech recognition model.

6. The interactive psychological assessment method according to claim 5, characterized in that: After the translated answer audio is converted into the corresponding answer text by the speech recognition model, the method further includes: Obtaining the real-time emotional state of the user; Dynamically adjusting the question settings in the DSM-5 psychological assessment question bank according to the real-time emotional state; The adjusted question is output to the user by driving the virtual character image voice.

7. The interactive psychological assessment method according to claim 1, characterized in that: The step of driving the virtual character image according to the question audio and the question text to transmit the question audio to the user comprises: The question audio and the question text are combined through a deep learning algorithm to generate an expression coefficient of the virtual character image; According to the expression coefficient, combined with 3D model binding and facial animation technology, the lip shape and facial expression of the virtual character are generated to drive the virtual character and transmit the question audio to the user.

8. An interactive psychological assessment device, characterized in that: The interactive psychological assessment device comprises: The question text conversion module is used to convert the question text in the DSM-5 psychological assessment question bank into the corresponding question audio through the text-to-speech model; An image driving module, used to drive the virtual character image according to the question audio and the question text, and transmit the question audio to the user; An answer audio conversion module is used to obtain the answer audio of the user based on the audio feedback of the question, and convert the answer audio into a corresponding answer text through a speech recognition model; An emotional state acquisition module, used to acquire a facial image of the user when answering, and obtain the emotional state of the user through a multimodal model based on the facial image and the answer audio; The empathy response module is used to drive the virtual character image to perform empathy response based on the emotional state and the answer text through a voice generation model.

9. An interactive psychological assessment device, characterized in that: The interactive psychological assessment device comprises: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the interactive psychological assessment device to perform the interactive psychological assessment method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the interactive psychological assessment method according to any one of claims 1 to 7 is implemented.

Images