Dual-mode online privacy protection psychological counseling system and method

Abstract

The invention provides a dual-mode online privacy protection psychological counseling system and method, and belongs to the field of psychological counseling and artificial intelligence, and the system comprises the steps: setting a data preparation module, a figure face conversion module, a tone conversion module and a data exchange module in a server, so as to complete the conversion of user face information into a target face, the user voice information is converted into user voice information of a target tone, the processed user voice information and user face information are sent to a psychologist terminal user interaction terminal, and the processed psychologist voice information and psychologist face information are sent to the user interaction terminal. According to the method, the trust sense and the security sense of the user are enhanced, so that the user can freely express feelings and problems without worrying about leakage of personal privacy, the psychological comfort of the user is improved, and more users sensitive to privacy are encouraged to actively seek psychological help.

Description

Technical Field

[0001] The present invention belongs to the field of psychological counseling and artificial intelligence, and specifically relates to a dual-mode online privacy protection psychological counseling system and method. Background Art

[0002] With the development of the Internet, online psychological counseling has gradually become an important mental health service model. Online psychological counseling aims to provide real-time remote psychological counseling services, so that users can get professional psychological help anytime and anywhere. However, when users are doing psychological counseling, they often have some concerns, which are mainly concentrated in several aspects: First, they are worried about the leakage of personal privacy, especially their facial images and voice information. The relative imperfections of privacy protection measures may cause users to have concerns about the use of personal information. Second, there is a lack of professional psychological counselors in application scenarios such as colleges and enterprises. Although these occasions promote mental health services, the quality of psychological counseling services is difficult to guarantee due to the shortage of professional talents. Third, the waiting time of users is wasted, the efficiency of psychological counseling is reduced, and the treatment effect is affected.

[0003] Some researchers have proposed new artificial intelligence technologies to conduct anonymous online interactions and optimize psychological counseling services. They mainly include: 1) Real-time face-changing technology: an effective privacy protection technology that can freely change faces in real time in videos while retaining the person's facial expressions; 2) Voice conversion technology: this technology can freely change the voice and simulate the voices of different people while retaining the original tone and speed.

[0004] At present, face-changing and voice-changing technologies are mainly used in the fields of entertainment, social networking and self-media, and are underutilized in professional fields such as psychological counseling. In addition, the research and development of the psychological counseling big model is in its initial stage and has not been effectively applied to the actual practice of auxiliary psychological counseling services. Usually, the model is only used to replace manual simple psychological counseling, but the problem that communication between people and machines can never achieve the effect of communication between people is ignored, resulting in reduced accuracy of psychological counseling. At present, there is no system that integrates real-time face-changing and voice-changing technology with the psychological counseling big model for online dual-mode psychological counseling combined with manual participation. Summary of the invention

[0005] In view of the deficiencies of the prior art, this application proposes a dual-mode online privacy protection psychological counseling system and method.

[0006] In the first aspect, the present application proposes a dual-mode online privacy protection psychological counseling system, which is applied to a server and includes: a data preparation module, a character face replacement module, a voice and color conversion module, and a data exchange module;

[0007] The data preparation module is connected to the character face changing module and the timbre conversion module respectively, and the character face changing module and the timbre conversion module are connected to the data exchange module respectively;

[0008] The data preparation module is used to obtain a target face and a target timbre from the interactive terminal, and send the target face to the character face replacement module, and send the target timbre to the timbre conversion module;

[0009] The data exchange module is used to receive the user voice information and user face information transmitted from the user interaction terminal, and receive the psychologist voice information and psychologist face information transmitted from the psychologist terminal; send the user voice information and psychologist voice information to the timbre conversion module, and receive the processed user voice information and psychologist voice information from the timbre conversion module; send the user face information and psychologist face information to the character face changing module, and receive the processed user face information and psychologist face information from the character face changing module; send the processed user voice information and user face information to the psychologist terminal, and send the processed psychologist voice information and psychologist face information to the user interaction terminal;

[0010] The character face-changing module is used to convert the user's face information into a target face to obtain the processed user's face information; convert the psychologist's face information into a target face to obtain the processed psychologist's face information;

[0011] The timbre conversion module is used to convert the user voice information into the user voice information of the target timbre to obtain the processed user voice information; and convert the psychologist voice information into the psychologist voice information of the target timbre to obtain the processed psychologist voice information.

[0012] The dual-mode online privacy protection psychological counseling system is applied to a server and further includes: a timbre training module;

[0013] The timbre training module is connected to the user interaction terminal and the timbre conversion module respectively, and is used to train the timbre material transmitted by the user interaction terminal with the timbre training model to obtain a new timbre weight, and send the new timbre weight to the timbre conversion module;

[0014] The timbre conversion module is also used to perform timbre conversion on the user voice information according to the new timbre weight to obtain the processed user voice information; and to perform timbre conversion on the psychologist voice information according to the new timbre weight to obtain the processed psychologist voice information.

[0015] The dual-mode online privacy protection psychological consultation system is applied to a server and further includes: an emotional dialogue module, a speech synthesis module and a face driving module;

[0016] The emotional dialogue module is connected to the user interaction terminal, the psychologist terminal and the speech synthesis module respectively, the speech synthesis module is connected to the face driving module, and the face driving module is connected to the user interaction terminal;

[0017] The emotional dialogue module is used to receive text information transmitted from the user interaction terminal when the emotional dialogue function is activated;

[0018] Inputting the text information into an emotional dialogue model to obtain an emotional dialogue text, and sending the emotional dialogue text to a speech synthesis module and a psychologist terminal;

[0019] The speech synthesis module is used to perform audio synthesis on the emotional dialogue text to obtain synthesized speech;

[0020] The face driving module is used to add the corresponding face driving to the synthesized speech to obtain the face driving video, and send the face driving video to the user interaction terminal.

[0021] The emotional dialogue module generates emotional dialogue text using an emotional dialogue model trained with a fine-tuning algorithm and a dialogue data set with a specific personality tone. The dialogue data set with a specific personality tone includes: multiple rounds of dialogues between doctors and users in a specific personality tone, wherein the multiple rounds of dialogues are obtained from a psychological counseling model, and the specific personality tone represents different doctor personality images.

[0022] The emotional dialogue model trained by using the fine-tuning algorithm and the dialogue data set with a specific personality and tone generates emotional dialogue text, and the training process includes:

[0023] Obtain a conversation dataset with a specific personality and tone;

[0024] Load the original fine-tuned weights of the tokenizer and InternLM model;

[0025] The word segmenter is used to convert multiple rounds of dialogue in the dialogue data set with a specific personality and tone into a vocabulary index, and the vocabulary index is padded with zeros to a set length to obtain a vocabulary index with a unified length;

[0026] Configure optimizer and learning rate scheduler;

[0027] The fine-tuning algorithm, configuration optimizer, and learning rate scheduler are used to re-fine-tune the original fine-tuned weights of the InternLM model according to the vocabulary index after the unified length to obtain the fine-tuned weights;

[0028] Merge the fine-tuned weights into the original fine-tuned weights of the InternLM model to get the new model weights;

[0029] The new model weight is used as the weight of the emotional dialogue model to obtain the trained emotional dialogue model, and the trained emotional dialogue model is used to generate emotional dialogue text.

[0030] The fine-tuning algorithm and the learning rate scheduler are used to re-fine-tune the original fine-tuning weights of the InternLM model to obtain the fine-tuned weights, including:

[0031] Calculate the loss function of the emotional dialogue model;

[0032] The gradient of the loss function is used to update the parameters of the low-rank matrix of the original fine-tuned weights of the InternLM model, where the cumulative step size of each gradient is calculated using the learning rate in the learning rate scheduler;

[0033] Based on the updated parameters, the LoRA method is used to fine-tune the original fine-tuned weights of the InternLM model to obtain the fine-tuned weights.

[0034] The loss function of the emotional dialogue model is calculated as follows:

[0035]

[0036] Among them, P(a i,j |·) is the emotional dialogue model that generates a response a given context and a specific character description i,j The probability of L is the loss function of the emotional dialogue model, q i,j is the jth question asked by the user in the i-th multi-round dialogue, Prompt k is the kth description of the doctor’s personality image, N is the total number of multi-round dialogues included in the dialogue dataset, and n i is the total number of rounds of multi-round dialogue between the user and the doctor in the i-th multi-round dialogue.

[0037] The cumulative step size of each gradient is calculated using the learning rate in the learning rate scheduler, and the calculation formula is as follows:

[0038]

[0039] Among them, η is the learning rate, θ t is the model parameter for the tth learning, θ t+1 is the parameter of the model during the t+1th learning, is the accumulated gradient, is the gradient of the loss function of the jth gradient accumulation, and k is the number of gradient accumulations.

[0040] The dual-mode online privacy protection psychological counseling system is applied to a user interaction terminal and includes:

[0041] The first camera is used to collect user face information and send the user face information to the data exchange module;

[0042] The first microphone is used to collect user voice information and send the user voice information to the data exchange module;

[0043] A face selection module is used to select a target face from a face sample library or upload a specified face image as a target face, and send the target face to a data preparation module;

[0044] A timbre selection module is used to select a target timbre from a sound sample library or upload a timbre material as a target timbre, and send the target timbre to a data preparation module;

[0045] The first receiving module is used to receive and display the processed user voice information and user face information.

[0046] The dual-mode online privacy protection psychological consultation system is applied to a psychologist terminal and includes:

[0047] The second camera is used to collect the facial information of the psychologist and send the facial information of the psychologist to the data exchange module;

[0048] The second microphone is used to collect the voice information of the psychologist and send the voice information of the psychologist to the data exchange module;

[0049] The second receiving module is used to receive and display the processed psychologist's voice information and psychologist's face information;

[0050] The third receiving module is used to provide professional psychological advice to the counselor using the emotional dialogue text when the counselor of the psychologist terminal is a non-professional.

[0051] In the second aspect, the present application proposes a dual-mode online privacy protection psychological counseling method, comprising:

[0052] Step S1: When the user and the psychologist are ready, go to step S2; when the user is ready but the psychologist is not, start the emotional dialogue function and go to step S8

[0053] Step S2: collecting user face information, user voice information, psychologist face information and psychologist voice information;

[0054] Step S3: Select a target face from the face sample library or upload a specified face image as the target face, and select a target timbre from the sound sample library or upload a timbre material as the target timbre;

[0055] Step S4: sending the user voice information and the psychologist voice information to the timbre conversion module for timbre conversion, converting the user voice and / or the psychologist voice information into user voice information of target timbre, and obtaining processed user voice information and psychologist voice information;

[0056] Step S5: sending the user's face information and the psychologist's face information to the character face-changing module, converting the user's face information and / or the psychologist's face information into a target face, and obtaining processed user's face information and psychologist's face information;

[0057] Step S6: sending the processed user voice information and user face information to the psychologist terminal;

[0058] Step S7: sending the processed psychologist voice information and psychologist face information to the user interaction terminal;

[0059] Step S8: When the emotional dialogue function is activated, receiving text information transmitted by the user interaction terminal;

[0060] Step S9: inputting the text information into the emotional dialogue model to obtain the emotional dialogue text;

[0061] Step S10: synthesizing the emotional dialogue text into audio to obtain synthesized speech;

[0062] Step S11: adding corresponding face drive to the synthesized speech to obtain a face drive video, and sending the face drive video to the user interaction terminal;

[0063] Step S12: Repeat steps S8 to S11 until the psychologist is ready, then go to step S2.

[0064] Beneficial effects:

[0065] This application proposes a dual-mode online privacy protection psychological counseling system and method, which uses real-time face-changing and voice-changing technology to protect the user's facial image and voice information and achieve anonymization. This advanced privacy protection enhances the user's sense of trust and security, allowing users to freely express their feelings and problems without worrying about the disclosure of personal privacy. It not only improves the user's psychological comfort, but also encourages more privacy-sensitive users to actively seek psychological help. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 Schematic diagram of the principle of the dual-mode online privacy protection psychological counseling system of the embodiment of the present application;

[0067] Figure 2 Flow chart of the dual-mode online privacy protection psychological counseling method according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to solve the privacy problem, this application supports face-changing and voice-changing synchronization technology, generates virtual identities for users and psychological counselors, and allows participants to communicate without revealing their true identities, thereby reducing the traditional model of psychological barriers to participating in online psychological counseling. In order to solve the problem of insufficient professional human resources, researchers have proposed a psychological counseling assistance technology based on a large model of psychological counseling. In this auxiliary mode, the psychological counseling large model can provide professional psychological counseling advice to non-professional psychological counselors based on user data and conversations, and provide psychological counseling services to patients. In order to solve the problem that waiting time for an appointment has not been effectively utilized, the system supports the provision of an intelligent emotional dialogue service to provide emotional support to users in the absence of manual services, and then connect to traditional psychological counseling or auxiliary psychological counseling.

[0069] The main purpose of this embodiment is to provide a dual-mode online privacy protection psychological counseling system based on a large psychological counseling model, aiming to improve the effect of online psychological counseling services and optimize user experience.

[0070] The system receives the user's voice and facial image information through wireless network communication between the cloud server and the user end. The cloud server transmits this data to the real-time voice conversion module and the character face replacement module respectively, and uses the deep learning model to anonymize the data and generate a virtual identity. The processed data is transmitted to the psychologist terminal to provide psychological counseling services in the traditional mode.

[0071] In addition, the system introduces an auxiliary mode to support deployment in non-professional environments (such as schools and enterprises). In the auxiliary mode, non-professional counselors (such as teachers, student counselors, and managers) can use the professional advice of the psychological counseling model (usually referred to as the psychological counseling big model in this field) to better provide psychological counseling services. The emotional dialogue model can generate appropriate dialogue content based on the user's emotions and dialogue status by training a large amount of text from the psychological counseling big model. The application of this mode can provide timely psychological help in an environment where professional psychologists are insufficient, and broaden the coverage of services, especially in remote areas or places with scarce resources. The psychological counseling big model enhances the professionalism of non-professional counseling. The psychological counseling big model integrates a wide range of psychological knowledge and actual cases, enabling non-professional counselors to provide professional and accurate counseling services without a deep psychological background. Through the professional advice generated by the system, non-professional counselors can help users more effectively while improving their own counseling skills and knowledge.

[0072] The relationship between the two can be expressed in simpler language: In particular, when the psychologist's terminal is busy and the user (i.e. the consultant) needs to wait, the system enables the emotional dialogue function. The user interacts with the emotional dialogue model in the cloud, and the text generated by the model is converted into audio and video output after audio synthesis and virtual face animation processing, providing real-time emotional support for the user.

[0073] This system not only provides privacy protection and reduces the psychological barriers of users when they are receiving psychological counseling, but also improves the acceptability and flexibility of counseling services through dual-mode design, especially in non-professional counseling scenarios, which significantly improves the coverage of services and user satisfaction. In addition, while users are waiting in line for psychological counseling, the system provides instant interaction through the emotional dialogue function, which effectively reduces the pressure and anxiety of users while waiting, and further improves the overall service quality and efficiency of the system.

[0074] Embodiment 1:

[0075] This embodiment proposes a dual-mode online privacy protection psychological consultation system, which is applied to a server such as Figure 1 As shown, it includes: a data preparation module, a character face changing module, a tone color conversion module and a data exchange module;

[0076] The data preparation module is connected to the character face changing module and the timbre conversion module respectively, and the character face changing module and the timbre conversion module are connected to the data exchange module respectively;

[0077] The data preparation module is used to obtain a target face and a target timbre from the interactive terminal, and send the target face to the character face replacement module, and send the target timbre to the timbre conversion module;

[0078] The data exchange module is used to receive the user voice information and user face information transmitted from the user interaction terminal, and receive the psychologist voice information and psychologist face information transmitted from the psychologist terminal; send the user voice information and psychologist voice information to the timbre conversion module, and receive the processed user voice information and psychologist voice information from the timbre conversion module; send the user face information and psychologist face information to the character face changing module, and receive the processed user face information and psychologist face information from the character face changing module; send the processed user voice information and user face information to the psychologist terminal, and send the processed psychologist voice information and psychologist face information to the user interaction terminal;

[0079] The character face-changing module is used to convert the user's face information into a target face to obtain the processed user's face information; convert the psychologist's face information into a target face to obtain the processed psychologist's face information;

[0080] The timbre conversion module is used to convert the user voice information into the user voice information of the target timbre to obtain the processed user voice information; and convert the psychologist voice information into the psychologist voice information of the target timbre to obtain the processed psychologist voice information.

[0081] The dual-mode online privacy protection psychological counseling system is applied to a server and further includes: a timbre training module;

[0082] The timbre training module is connected to the user interaction terminal and the timbre conversion module respectively, and is used to train the timbre material transmitted by the user interaction terminal with the timbre training model to obtain a new timbre weight, and send the new timbre weight to the timbre conversion module;

[0083] The timbre conversion module is also used to perform timbre conversion on the user voice information according to the new timbre weight to obtain the processed user voice information; and to perform timbre conversion on the psychologist voice information according to the new timbre weight to obtain the processed psychologist voice information.

[0084] The dual-mode online privacy protection psychological consultation system is applied to a server and further includes: an emotional dialogue module, a speech synthesis module and a face driving module;

[0085] The emotional dialogue module is connected to the user interaction terminal, the psychologist terminal and the speech synthesis module respectively, the speech synthesis module is connected to the face driving module, and the face driving module is connected to the user interaction terminal;

[0086] The emotional dialogue module is used to receive text information transmitted from the user interaction terminal when the emotional dialogue function is activated;

[0087] Inputting the text information into an emotional dialogue model to obtain an emotional dialogue text, and sending the emotional dialogue text to a speech synthesis module and a psychologist terminal;

[0088] The speech synthesis module is used to perform audio synthesis on the emotional dialogue text to obtain synthesized speech;

[0089] The face driving module is used to add the corresponding face driving to the synthesized speech to obtain the face driving video, and send the face driving video to the user interaction terminal.

[0090] The dual-mode online privacy protection psychological counseling system is applied to a user interaction terminal and includes:

[0091] The first camera is used to collect user face information and send the user face information to the data exchange module;

[0092] The first microphone is used to collect user voice information and send the user voice information to the data exchange module;

[0093] A face selection module is used to select a target face from a face sample library or upload a specified face image as a target face, and send the target face to a data preparation module;

[0094] A timbre selection module is used to select a target timbre from a sound sample library or upload a timbre material as a target timbre, and send the target timbre to a data preparation module;

[0095] The first receiving module is used to receive and display the processed user voice information and user face information.

[0096] The dual-mode online privacy protection psychological consultation system is applied to a psychologist terminal and includes:

[0097] The second camera is used to collect the facial information of the psychologist and send the facial information of the psychologist to the data exchange module;

[0098] The second microphone is used to collect the voice information of the psychologist and send the voice information of the psychologist to the data exchange module;

[0099] The second receiving module is used to receive and display the processed psychologist's voice information and psychologist's face information;

[0100] The third receiving module is used to provide professional psychological advice to the counselor using the emotional dialogue text when the counselor of the psychologist terminal is a non-professional.

[0101] In this embodiment, the dual-mode online privacy protection psychological counseling system includes: a cloud server (S) and two interactive terminals. According to the user identity, the two interactive terminals are divided into a user interactive terminal (T1) and a psychologist terminal (T2). Among them, the deep learning model is deployed on the cloud server. The audio and video interactive interface is deployed on the two interactive terminals. The cloud server and the interactive terminal communicate with each other through audio and video streams.

[0102] The interactive terminals are all computer devices equipped with cameras and microphones. Their main tasks are: (1) Provide users with sample faces for replacement. When users want to use other faces, they provide customized face services for users to upload face pictures. (2) Provide users with sample voices for replacement. When users want to use other voices, they provide customized voice services for users to upload voice training materials. (3) Collect users' audio and video data and send them to the cloud server. (4) Receive audio and video data streams sent from the cloud server and present them to users. For the user interactive terminal (T1), the received data is the audio and video data of the psychologist after the face and voice are changed. For the psychologist terminal (T2), the received data is the audio and video data of the user (i.e., the consultant) after the face and voice are changed. (5) In particular, for the user interactive terminal (T1), when the psychologist is occupied and needs to queue, the emotional dialogue function is provided for users. (6) In particular, for the psychologist terminal, when the counselor is a non-professional, such as a student counselor or factory manager, the psychological counseling big language model is used to provide professional psychological advice to the non-professional counselor during the counseling process.

[0103] The cloud server includes: data preparation module, character face-changing module, timbre conversion module (this timbre conversion module is real-time timbre conversion), data exchange module, timbre training module, emotional dialogue module, speech synthesis module and face driving module. The main tasks are: (1) The data preparation module receives the target face and target timbre from the interactive terminal. When there is a need for custom timbre, it receives the timbre material from the interactive terminal. (2) After the conversation starts, the data exchange module receives the video stream and audio stream data transmitted by the interactive terminal in real time, and transmits the processed video stream and audio stream to another interactive terminal. (3) Use the character face-changing module to perform character face-changing on the received original video stream. (4) Use the real-time timbre conversion module to convert the received audio stream data into other timbre. (5) Use the timbre training module to train the model of the timbre material uploaded by the user to obtain a new timbre weight. (6) Use the emotional dialogue module to conduct emotional dialogue with the user (i.e. the consultant) while waiting in line, and provide temporary comfort to the user while waiting. (7) Use the speech synthesis module to convert the text generated by the emotional dialogue module into speech output. (8) Use the face driving module to drive the audio output by the speech synthesis module to achieve the effect of face-to-face conversation with people. (9) Use the emotional dialogue module to provide professional psychological counseling advice to non-professional counselors.

[0104] In this embodiment, the specific working process is as follows:

[0105] (1) Face and voice selection: The interactive interface provides several prepared face images and voices for users to choose from. For the custom face function, users need to upload face images. For the custom voice function, users need to upload voice training materials to the cloud server for training. The format of the voice training materials is: a zip file containing a folder. There are several audio files in the folder. The total audio length is about one minute. Try to ensure that there is no excessive noise in the audio file.

[0106] (2) Audio and data stream collection: After the conversation begins, the microphone and camera collect the user's voice and video in real time and upload them to the cloud server. The video data and audio data are transmitted asynchronously, with the video data transmission protocol being UDP / IP and the audio transmission protocol being TCP / IP.

[0107] (3) Audio and video stream reception: After the conversation begins, the cloud server receives the audio and video stream data from the other interactive terminal. Video and audio data are also transmitted asynchronously. The video data transmission protocol is UDP / IP, and the audio transmission protocol is TCP / IP.

[0108] (4) Emotional dialogue: When users (i.e. consultants) need to wait in line, an emotional dialogue function is provided for them. Users interact with the emotional dialogue model deployed on the cloud server. The text generated by the emotional dialogue model is presented to the user (i.e. consultant) in the form of audio and video after audio synthesis and face driving, providing temporary comfort for them.

[0109] (5) Consultation assistance: In particular, for psychologist terminals, when the consultation environment is in a non-professional environment such as a school or factory, where there is a lack of psychologists and non-professionals such as teachers, counselors, and management are needed for consultation, the psychological consultation big language model is used to provide advice to non-professional counselors during the consultation process: the system will collect the audio during the consultation process in real time, and send the recognized text to the cloud server after voice recognition. At the same time, the text generated by the psychological consultation big model returned by the server is received to provide professional advice to the counselor.

[0110] The specific workflow of each module is as follows:

[0111] (1) Data preparation module: Receive the target face and target voice from the user interaction terminal (T1) and the psychologist terminal (T2) respectively. If the target voice of a certain interaction terminal is a custom voice, receive voice training materials from the interaction terminal.

[0112] (2) Data exchange module: Receives real-time video and audio data from the user interaction terminal (T1) and the psychologist terminal (T2) respectively. The video stream data received from the user interaction terminal (T1) is denoted as V1r , audio stream data is recorded as A1 r The video stream data received from the psychologist terminal (T2) is recorded as V2 r , audio stream data is recorded as A2 r . V1 r With V2 r All are sent to the character face replacement module for processing, A1 r With A2 r All are sent to the real-time tone conversion module for processing. Video stream V1 r After processing, it is recorded as V1 s , audio stream A1 r After processing, it is recorded as A1 s . V1 s With A1 s All are sent to the psychologist terminal (T2). Video stream V2 r After processing, it is recorded as V2 s , audio stream A2 r After processing, it is recorded as A2 s . V2 s With A2 s All are sent to the user interaction terminal (T1). In the above data transmission process, the video stream and the audio stream are transmitted asynchronously. Among them, the video stream transmission protocol is UDP / IP; the audio stream transmission protocol is TCP / IP.

[0113] (3) Face-changing module: Obtain video stream V1 from the data exchange module r With V2 r . Use facefusion technology to perform face swapping operations on each frame of the video stream. For the image processing of each frame, the specific method is as follows: obtain the current image frame from the video stream, recorded as frame1, and obtain the target replacement face image from the data preparation module, recorded as frame2. Use the deep learning project insightface, based on the buffalo_l model, and use face analysis to perform face detection processing on frame1 and frame2 respectively, obtain the face object from frame1, recorded as face1, and obtain the face from frame2, recorded as face2. After the above operations are completed, use the inswapper_128.onnx model to perform face swapping operations on the face objects face1 and face2 in the frame1 image to obtain the result image, recorded as frame3, and the image frame processing is completed. For the video stream V1 r With V2 r Each frame in is processed using the above method to obtain video stream V1 s With V2 s and returns it to the data exchange module.

[0114] (4) Tone conversion module: obtains audio stream A1 from the data exchange module r With A2 r . Use the Retrieval-based-Voice-Conversion technology to process the audio stream. The specific method is as follows: For the current audio stream, take 10,000 bytes as the standard length and intercept the audio frame. From the data preparation module, obtain the timbre to be converted and the corresponding pre-trained timbre weight file. For each audio frame, perform timbre conversion based on the huber model and the trained timbre weight. For audio stream A1 r With A2 r Each frame is processed using the above method to obtain audio stream A1 s With A2 s and returns it to the data exchange module.

[0115] (5) Tone training module: Obtain the audio training materials uploaded by the user from the data preparation module. After decompressing the materials, traverse all audio files in the training folder and perform slice normalization. Use the rvmpe algorithm to extract the pitch of the audio and obtain the audio feature file. Based on the above processing results, use the RVCv2 base model to perform model training and obtain the tone weight file. After the training is completed, the tone weight file is returned to the data preparation module.

[0116] (6) Emotional dialogue module: collects multi-round dialogue sets of psychological consultation in real life U = {D1, D2, ..., D m}, where i∈(1,2,...,m), D i is the i-th multi-round dialogue, n i is the number of rounds in the i-th multi-round dialogue, q i,j represents the jth question asked by the user in the i-th multi-round dialogue, a i,j Indicates the doctor's jth answer in the i-th multi-round dialogue. Set X descriptions of the doctor's personality image, where Prompt k represents the kth description of the doctor's personality image, and then extracts all the questions asked by the user in the i-th multi-round dialogue set, and merges them with the kth description of the doctor's personality image into a new question vector k, in According to the k-th description of the doctor's personality image, we get the question vector set So we got the question set Then the question vector k in the question vector set is As input, we use GPT-4, Wenxin Yiyan and Tongyi Qianwen to conduct multiple rounds of dialogue respectively, and use ChatGLM to integrate the answers of the first three models to obtain a new multi-round dialogue set. The new multi-round dialogue i, The integrated model answer for the jth question in the i-th multi-round dialogue. Then, the internLM model (International Language Model) is used as the base model, and the multi-round dialogue set As a fine-tuning dataset, the base model is then fine-tuned using the Xtuner tool to obtain the kth emotional dialogue language model with a specific doctor's personality image. When performing text generation tasks, a doctor's personality image is randomly selected, and professional psychological counseling assistance is provided based on the input text. For different types of generation tasks, different prompts are given before the generation starts to adapt to the task scenario.

[0117] (7) Speech synthesis module: The generated emotional dialogue text T is obtained from the emotional dialogue module. The python library langdetect is used to perform language detection on the text to obtain the language type detection result L. In order to ensure the speech synthesis effect, the text T is preprocessed. The specific process is as follows: Use regular expressions to divide the text T into sentences, and finally obtain n sub-texts. After that, all sub-texts are sorted and combined into m text segments with a length of no more than 80 characters. After the preprocessing is completed, for each text, according to the detection result L, the TTS module is used based on the xtts_v2 model to perform speech synthesis to obtain m audio files. Finally, all audio files are spliced ​​to obtain the final processing result audio W. After the processing is completed, the audio W is passed to the face driving module.

[0118] (8) Face driving module: The audio W is obtained from the speech synthesis module. Several face images are preset in this module. One face image is randomly selected from the face images and is recorded as P. Using DreamTalk technology, the audio W and the image P are used to generate a video V. The content of the video V is that the audio W drives the image P, so that the face in the image P speaks, and the mouth shape of the face is the same as the text in the audio W. After the processing is completed, the video V is returned to the user interaction terminal.

[0119] The collaboration scheme between the user interaction terminal (T1), the psychologist terminal (T2) and the cloud server is as follows:

[0120] The user interaction terminal (T1), the psychologist terminal (T2) and the cloud server communicate in real time. The cloud server is responsible for processing complex tasks and algorithms, while the user interaction terminal (T1) and the psychologist terminal (T2) are responsible for collecting audio and video signals. The three realize the function of privacy protection psychological counseling through collaboration. Specifically, when the user interaction terminal (T1) and the psychologist terminal (T2) interact, they use microphones and cameras to collect audio and video stream information and transmit them to the cloud server. Among them, the video stream information uses the UDP / IP protocol, and the audio stream information uses the TCP / IP protocol. In the cloud server, the data exchange module is responsible for receiving and transmitting information. The character face replacement module replaces the face in the received video stream with the target face. The audio conversion module converts the timbre of the received audio stream into the target timbre. When the user (i.e. the consultant) needs to wait, the cloud server and the interaction terminal provide emotional dialogue function. The text generated by the emotional dialogue module is presented to the user (i.e. the consultant) in the form of video after being processed by the audio synthesis module and the face driving module. When the psychologist is a non-professional counselor, the cloud server provides professional advice to the non-professional counselor.

[0121] Fine-tuning technology plays a key role in the field of deep learning, especially in emotional dialogue systems, which allows researchers to optimize models for specific tasks, such as emotion recognition and response generation, while maintaining the extensive knowledge of pre-trained models. At present, widely used fine-tuning algorithms include LoRA (Low-Rank Adaptation) and QLoRA (Quantized Low-Rank Adaptation), which mainly use low matrix decomposition and introduce low-rank adaptation layers to adjust the weights of pre-trained models without significantly increasing the number of parameters. The X tuner tool integrates these algorithms and uses algorithm acceleration technology to improve the efficiency of the fine-tuning process. On this basis, this embodiment explores the innovative application of dataset customization and hyperparameter adaptive adjustment in emotional dialogue.

[0122] This embodiment makes three improvements on the emotional dialogue function: First, by adopting a dialogue dataset with a specific personality tone, the performance of the model in emotional dialogue processing is significantly improved. Secondly, this embodiment designs a set of hyperparameter adjustment methods including learning rate, batch size, training cycle and regularization coefficient to optimize the model training of the emotional dialogue system and ensure that the model achieves the best performance on specific tasks. In addition, in order to improve training efficiency, this embodiment integrates the application of gradient accumulation technology. By accumulating smaller batches of gradients to simulate the effect of large-batch training, a more stable and efficient training process is achieved.

[0123] The improved fine-tuning strategy is not only innovative in engineering, but also demonstrates significant performance improvements in practical applications, especially in improving the response quality and emotion recognition capabilities of emotional dialogue systems.

[0124] The emotional dialogue module generates emotional dialogue text using an emotional dialogue model trained with a fine-tuning algorithm and a dialogue data set with a specific personality tone. The dialogue data set with a specific personality tone includes: multiple rounds of dialogues between doctors and users in a specific personality tone, wherein the multiple rounds of dialogues are obtained from a psychological counseling model, and the specific personality tone represents different doctor personality images.

[0125] For example, a specific personality tone can be a father's serious personality or a mother's gentle and considerate personality. In order to construct the conversation data of a specific personality tone, in this embodiment, it is necessary to first determine a specific personality image, that is, the doctor's personality image, and then generate a corresponding conversation data set of a specific personality tone based on this image. The conversation data set of a specific personality tone consists of multiple rounds of conversations between doctors and users. The doctor's answer should be consistent with the specific personality tone.

[0126] The emotional dialogue module uses a fine-tuning algorithm and is trained based on a dialogue dataset containing different emotional states and personality traits to generate emotional dialogue text. For example, the dataset includes dialogues in different emotional states such as anger, happiness, anxiety, fear, and language styles of different personality types such as extroversion and introversion.

[0127] The emotional dialogue model trained by using the fine-tuning algorithm and the dialogue data set with a specific personality and tone generates emotional dialogue text, and the training process includes:

[0128] Step S101: obtaining a dialogue dataset with a specific personality and tone;

[0129] Step S102: Load the original fine-tuning weights of the word segmenter and the InternLM model;

[0130] Step S103: using the word segmenter to convert multiple rounds of dialogue in the dialogue data set with a specific personality and tone into vocabulary indexes, and padding the vocabulary indexes with zeros to a set length to obtain vocabulary indexes with a unified length;

[0131] Step S104: configuring the optimizer and the learning rate scheduler;

[0132] Step S105: using a fine-tuning algorithm, a configuration optimizer, and a learning rate scheduler, re-fine-tune the original fine-tuning weights of the InternLM model according to the vocabulary index after the unified length, to obtain the fine-tuned weights;

[0133] Step S106: merging the fine-tuned weights into the original fine-tuned weights of the InternLM model to obtain new model weights;

[0134] Step S107: adopting the new model weight as the weight of the emotional dialogue model to obtain the trained emotional dialogue model, and adopting the trained emotional dialogue model to generate the emotional dialogue text.

[0135] The fine-tuning algorithm and the learning rate scheduler are used to re-fine-tune the original fine-tuning weights of the InternLM model to obtain the fine-tuned weights, including:

[0136] Step S105.1: Calculate the loss function of the emotional dialogue model;

[0137] Step S105.2: Use the gradient of the loss function to update the parameters of the low-rank matrix of the original fine-tuning weights of the InternLM model, where the cumulative step size of each gradient is calculated using the learning rate in the learning rate scheduler;

[0138] Step S105.3: Based on the updated parameters, the LoRA method is used to fine-tune the original fine-tuning weights of the InternLM model to obtain the fine-tuned weights.

[0139] In this embodiment, the fine-tuning process is mainly divided into the following five steps: (1) loading the model and fine-tuning data; (2) data preprocessing; (3) configuring the optimizer and learning rate scheduler; (4) fine-tuning training; and (6) merging parameters.

[0140] The steps of loading models and fine-tuning data are to load the weights that can be fine-tuned in the word segmenter and the internLM model (InternationalModel), and to load the new multi-round dialogue set U′ for fine-tuning. k .

[0141] The data preprocessing step is to use a word segmenter to transform the input data set U′ k The multi-round dialogue in is converted into a vocabulary index. Among them, a specific personality tone corresponds to a multi-round dialogue between a doctor and a user. For the sub-elements in the dataset, that is, the new multi-round dialogue Let each reply or question be x and convert it into a vocabulary index sequence token(x) = [t1, t2, …, t len(x) ], where t i Indicates the index of the i-th word in x in the vocabulary, and then pads the index sequence to the set maximum length L, that is, pad(token(x)) = [t1, t2, …, t len(x) ,0,…,0].

[0142] The configuration optimizer and learning rate scheduler step is to select AdamW as the optimizer. In this embodiment, the AdamW optimizer is used in conjunction with the learning rate scheduler to control the speed of model learning. This embodiment uses optim.AdamW in PyTorch to initialize the AdamW optimizer, and sets the parameter to η=1×10 -5 ,β1=0.9,β2=0.999,λ=0.0001, and then configure the linear warmup and cosine annealing learning rate scheduler as follows:

[0143]

[0144] Among them, t is the current number of training rounds, η0 is the initial learning rate, η t is the learning rate of the tth round, η min is the minimum learning rate, t warmup is the number of warm-up steps, and T is the total number of training steps.

[0145] The fine-tuning training step is to select the LoRA method to fine-tune the model weights. Let the weight matrix W = A × B, where A and B are low-rank matrices. In each training step, A and B are updated, thereby updating W. The parameter update formula is ΔW = A × ΔB + ΔA × B. The total loss function L during the training process is defined as L:

[0146]

[0147] Among them, P(a i,j |·) is the model generating the answer a given context and specified personality description. i,j During back propagation, the gradient of the loss function is calculated In the training step, the gradient accumulation technique is used to perform multi-step gradient accumulation k times with a smaller batch size, and then update the parameters. The formula is as follows: in, is the gradient of the loss function of the jth gradient accumulation. In each gradient accumulation step, Among them, η is the learning rate, θ t are model parameters, is the accumulated gradient. Then the currently trained model parameters are saved periodically, and the model generation performance is evaluated at a predetermined evaluation frequency.

[0148] The step of merging parameters is to merge the fine-tuned parameters in the internlm model into the original model weights to form new model weights.

[0149] Embodiment 2:

[0150] This application proposes a dual-mode online privacy-preserving psychological counseling method, such as Figure 2 As shown, including:

[0151] Step S1: When the user and the psychologist are ready, go to step S2; when the user is ready but the psychologist is not, start the emotional dialogue function and go to step S8

[0152] Step S2: collecting user face information, user voice information, psychologist face information and psychologist voice information;

[0153] Step S3: Select a target face from the face sample library or upload a specified face image as the target face, and select a target timbre from the sound sample library or upload a timbre material as the target timbre;

[0154] Step S4: sending the user voice information and the psychologist voice information to the timbre conversion module for timbre conversion, converting the user voice and / or the psychologist voice information into user voice information of target timbre, and obtaining processed user voice information and psychologist voice information;

[0155] Step S5: sending the user's face information and the psychologist's face information to the character face-changing module, converting the user's face information and / or the psychologist's face information into a target face, and obtaining processed user's face information and psychologist's face information;

[0156] Step S6: sending the processed user voice information and user face information to the psychologist terminal;

[0157] Step S7: Send the processed psychologist's voice information and psychologist's face information to the user interaction terminal.

[0158] The dual-mode online privacy protection psychological counseling method further includes:

[0159] Step S8: When the emotional dialogue function is activated, receiving text information transmitted by the user interaction terminal;

[0160] Step S9: inputting the text information into the emotional dialogue model to obtain the emotional dialogue text;

[0161] Step S10: synthesizing the emotional dialogue text into audio to obtain synthesized speech;

[0162] Step S11: adding corresponding face drive to the synthesized speech to obtain a face drive video, and sending the face drive video to the user interaction terminal;

[0163] Step S12: Repeat steps S8 to S11 until the psychologist is ready, then go to step S2.

[0164] When the emotional dialogue function is activated, the specific process is as follows:

[0165] (1-1) The cloud server is initialized and two queues, queue1 and queue2, are created. When the psychologist is ready, he enters queue2. When the user (i.e., the consultant) is ready, he enters queue1. When queue1 is not empty, if queue2 is not empty, the first user of queue1 is paired with the first user of queue2, and the two are dequeued, and the process goes to step (1-2). If queue2 is empty, there is no psychologist ready to provide emotional dialogue services to the user (i.e., the consultant), and the process goes to step (2-1);

[0166] (1-2) After the user (i.e., consultant) is successfully matched with the psychologist, the user (i.e., consultant) selects the face and voice that he / she wants to replace on the user interaction terminal (T1). The user (i.e., consultant) can choose to customize the face and voice. If you customize the face, you need to upload an image with a frontal face. If you customize the voice, you need to upload the timbre training material, which is a zip file containing a voice file with a total duration of about 1 minute. After the selection is completed, the user interaction terminal (T1) transmits the face and voice files to the cloud server. The psychologist directly selects from the preset faces and voices on the psychologist terminal (T2), and the selection results are transmitted to the cloud server;

[0167] (1-3) The data preparation module of the cloud server receives the selected face and voice from the user interaction terminal (T1) and the psychologist terminal (T2). If there is a custom voice that needs to be trained, the voice training material is sent to the voice training module for training, and the trained voice is returned to the data preparation module. When the data preparation is completed, the data preparation module passes the face to be replaced to the character face replacement module, and the voice to be converted to the voice conversion module. A start signal is sent to the user interaction terminal (T1) and the psychologist terminal (T2) to start the psychological counseling process and enter (1-4); if the psychologist is a non-professional counselor, step (1-5) is executed at the same time.

[0168] (1-4) After the user interaction terminal (T1) and the psychologist terminal (T2) receive the start signal, the cameras and microphones of the two terminals begin to collect video stream and audio stream information and transmit it to the cloud server. For the data of any interactive terminal, after the data exchange module of the cloud server receives the video stream and audio stream information, it passes the video stream to the character face changing module for face changing processing, and passes the audio stream information to the timbre conversion module for voice changing processing. After the processed results are returned to the data exchange module, they are sent to the other terminal. For any terminal, after receiving the audio and video information processed by the other terminal, it is directly presented to the user. This step is repeated until the psychological counseling process is completed.

[0169] (1-5) During the consultation process, the user interaction terminal will continuously record the user's (i.e. the consultant's) voice data and convert it into text through voice recognition. The text is sent to the cloud server. The cloud server uses the psychological consultation language model to generate professional consultation suggestions and send them to the psychologist terminal for reference.

[0170] For the emotional dialogue process, the specific steps are as follows:

[0171] (2-1) The user interaction terminal enters the emotional dialogue interaction interface. The user (i.e., the consultant) enters the dialogue content in the terminal, and the terminal sends the dialogue content to the cloud server.

[0172] (2-2) After receiving the conversation content of the user (i.e., the consultant), the cloud server passes it to the emotional dialogue module. The emotional dialogue module inputs the conversation content into the psychological counseling language model to obtain the generated language text with psychological help properties, and passes the language text to the speech synthesis module.

[0173] (2-3) After receiving the language text, the speech synthesis module performs language detection, sentence segmentation, and organization on the text. Several text segments are obtained. Speech synthesis technology is used for all text segments, and the synthesized audio is spliced ​​to obtain the final result audio. After the processing is completed, the result audio is passed to the face driving module.

[0174] (2-4) The face driving module drives the face image preset in the module with the received audio result to obtain the audio-driven face dialogue video. The final video is returned to the user interaction terminal and presented to the user (ie, the consultant).

[0175] (2-4) Repeat steps (2-1) to (2-4) until a psychologist is ready.

[0176] The various embodiments in the present application are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0177] In this embodiment, enabling the emotional dialogue function helps improve user satisfaction and service quality: when users need to wait in line, the emotional dialogue function provided by the system can interact with users in a timely manner, reducing users' anxiety and waiting pressure. In addition, the dual-mode design of the system also improves the flexibility and practicality of psychological consultation. Through this personalized and immediate support, user satisfaction and overall service efficiency are significantly improved.

[0178] The protection scope of the present application is not limited to the above-mentioned embodiments. Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the scope and spirit of the present disclosure. If these changes and modifications fall within the scope of the claims of the present disclosure and their equivalents, the intention of the present disclosure also includes these changes and modifications.

Claims

1. A dual-mode online privacy protection psychological counseling system, applied to a server, characterized in that: include: Data preparation module, character face changing module, voice and color conversion module and data exchange module; The data preparation module is connected to the character face changing module and the timbre conversion module respectively, and the character face changing module and the timbre conversion module are connected to the data exchange module respectively; The data preparation module is used to obtain a target face and a target timbre from the interactive terminal, and send the target face to the character face replacement module, and send the target timbre to the timbre conversion module; The data exchange module is used to receive the user voice information and user face information transmitted from the user interaction terminal, and receive the psychologist voice information and psychologist face information transmitted from the psychologist terminal; Send the user voice information and the psychologist voice information to the timbre conversion module, and receive the processed user voice information and the psychologist voice information from the timbre conversion module; send the user face information and the psychologist face information to the character face changing module, and receive the processed user face information and the psychologist face information from the character face changing module; send the processed user voice information and the user face information to the psychologist terminal, and send the processed psychologist voice information and the psychologist face information to the user interaction terminal; The character face-changing module is used to convert the user's face information into a target face to obtain the processed user's face information; convert the psychologist's face information into a target face to obtain the processed psychologist's face information; The timbre conversion module is used to convert the user voice information into the user voice information of the target timbre to obtain the processed user voice information; and convert the psychologist voice information into the psychologist voice information of the target timbre to obtain the processed psychologist voice information.

2. The dual-mode online privacy protection psychological counseling system according to claim 1 is characterized in that: The dual-mode online privacy protection psychological counseling system further includes: a timbre training module; The timbre training module is connected to the user interaction terminal and the timbre conversion module respectively, and is used to train the timbre material transmitted by the user interaction terminal with the timbre training model to obtain a new timbre weight, and send the new timbre weight to the timbre conversion module; The timbre conversion module is also used to perform timbre conversion on the user voice information according to the new timbre weight to obtain the processed user voice information; and to perform timbre conversion on the psychologist voice information according to the new timbre weight to obtain the processed psychologist voice information.

3. The dual-mode online privacy protection psychological counseling system according to claim 1 is characterized in that: The dual-mode online privacy protection psychological counseling system also includes: an emotional dialogue module, a speech synthesis module and a face driving module; The emotional dialogue module is connected to the user interaction terminal, the psychologist terminal and the speech synthesis module respectively, the speech synthesis module is connected to the face driving module, and the face driving module is connected to the user interaction terminal; The emotional dialogue module is used to receive text information transmitted from the user interaction terminal when the emotional dialogue function is activated; Inputting the text information into an emotional dialogue model to obtain an emotional dialogue text, and sending the emotional dialogue text to a speech synthesis module and a psychologist terminal; The speech synthesis module is used to perform audio synthesis on the emotional dialogue text to obtain synthesized speech; The face driving module is used to add the corresponding face driving to the synthesized speech to obtain the face driving video, and send the face driving video to the user interaction terminal.

4. The dual-mode online privacy protection psychological counseling system according to claim 2 is characterized in that: The emotional dialogue module generates emotional dialogue text using an emotional dialogue model trained with a fine-tuning algorithm and a dialogue data set with a specific personality tone. The dialogue data set with a specific personality tone includes: multiple rounds of dialogues between doctors and users in a specific personality tone, wherein the multiple rounds of dialogues are obtained from a psychological counseling model, and the specific personality tone represents different doctor personality images.

5. The dual-mode online privacy protection psychological counseling system according to claim 4 is characterized in that: The emotional dialogue model trained by using the fine-tuning algorithm and the dialogue data set with a specific personality and tone generates emotional dialogue text, and the training process includes: Obtain a conversation dataset with a specific personality and tone; Load the original fine-tuned weights of the tokenizer and InternLM model; The word segmenter is used to convert multiple rounds of dialogue in the dialogue data set with a specific personality and tone into a vocabulary index, and the vocabulary index is padded with zeros to a set length to obtain a vocabulary index with a unified length; Configure optimizer and learning rate scheduler; The fine-tuning algorithm, configuration optimizer, and learning rate scheduler are used to re-fine-tune the original fine-tuned weights of the InternLM model according to the vocabulary index after the unified length to obtain the fine-tuned weights; Merge the fine-tuned weights into the original fine-tuned weights of the InternLM model to get the new model weights; The new model weight is used as the weight of the emotional dialogue model to obtain the trained emotional dialogue model, and the trained emotional dialogue model is used to generate emotional dialogue text.

6. The dual-mode online privacy protection psychological counseling system according to claim 5, characterized in that: The fine-tuning algorithm and the learning rate scheduler are used to re-fine-tune the original fine-tuning weights of the InternLM model to obtain the fine-tuned weights, including: Calculate the loss function of the emotional dialogue model; The gradient of the loss function is used to update the parameters of the low-rank matrix of the original fine-tuned weights of the InternLM model, where the cumulative step size of each gradient is calculated using the learning rate in the learning rate scheduler; Based on the updated parameters, the LoRA method is used to fine-tune the original fine-tuned weights of the InternLM model to obtain the fine-tuned weights.

7. The dual-mode online privacy protection psychological counseling system according to claim 6 is characterized in that: The loss function of the emotional dialogue model is calculated as follows: Among them, P(a i,j |·) is the emotional dialogue model that generates a response a given context and a specific character description i,j The probability of L is the loss function of the emotional dialogue model, q i,j is the jth question asked by the user in the i-th multi-round dialogue, Prompt k is the kth description of the doctor’s personality image, N is the total number of multi-round dialogues included in the dialogue dataset, and n i is the total number of rounds of multi-round dialogues between the user and the doctor in the i-th multi-round dialogue; The cumulative step size of each gradient is calculated using the learning rate in the learning rate scheduler, and the calculation formula is as follows: Among them, η is the learning rate, θ t is the model parameter for the tth learning, θ t+1 is the parameter of the model during the t+1th learning, is the accumulated gradient, is the gradient of the loss function of the jth gradient accumulation, and k is the number of gradient accumulations.

8. A dual-mode online privacy protection psychological counseling system, applied to a user interaction terminal, characterized in that: include: The first camera is used to collect user face information and send the user face information to the data exchange module; The first microphone is used to collect user voice information and send the user voice information to the data exchange module; A face selection module is used to select a target face from a face sample library or upload a specified face image as a target face, and send the target face to a data preparation module; A timbre selection module is used to select a target timbre from a sound sample library or upload a timbre material as a target timbre, and send the target timbre to a data preparation module; The first receiving module is used to receive and display the processed user voice information and user face information.

9. A dual-mode online privacy protection psychological consultation system, applied to a psychologist terminal, characterized in that: include: The second camera is used to collect the facial information of the psychologist and send the facial information of the psychologist to the data exchange module; The second microphone is used to collect the voice information of the psychologist and send the voice information of the psychologist to the data exchange module; The second receiving module is used to receive and display the processed psychologist's voice information and psychologist's face information; The third receiving module is used to provide professional psychological advice to the counselor using the emotional dialogue text when the counselor of the psychologist terminal is a non-professional.

10. A dual-mode online privacy protection psychological counseling method, characterized in that: include: Step S1: When the user and the psychologist are ready, go to step S2; when the user is ready but the psychologist is not, start the emotional dialogue function and go to step S8 Step S2: collecting user face information, user voice information, psychologist face information and psychologist voice information; Step S3: Select a target face from the face sample library or upload a specified face image as the target face, and select a target timbre from the sound sample library or upload a timbre material as the target timbre; Step S4: sending the user voice information and the psychologist voice information to the timbre conversion module for timbre conversion, converting the user voice and / or the psychologist voice information into user voice information of target timbre, and obtaining processed user voice information and psychologist voice information; Step S5: sending the user's face information and the psychologist's face information to the character face-changing module, converting the user's face information and / or the psychologist's face information into a target face, and obtaining processed user's face information and psychologist's face information; Step S6: sending the processed user voice information and user face information to the psychologist terminal; Step S7: sending the processed psychologist voice information and psychologist face information to the user interaction terminal; Step S8: When the emotional dialogue function is activated, receiving text information transmitted by the user interaction terminal; Step S9: inputting the text information into the emotional dialogue model to obtain the emotional dialogue text; Step S10: synthesizing the emotional dialogue text into audio to obtain synthesized speech; Step S11: adding corresponding face drive to the synthesized speech to obtain a face drive video, and sending the face drive video to the user interaction terminal; Step S12: Repeat steps S8 to S11 until the psychologist is ready, then go to step S2.

Images