Psychological consultation auxiliary system and psychological consultation method

Abstract

The invention provides a psychological counseling auxiliary system. The system comprises a psychological analysis module, a psychological counseling resource library, a recommendation module and an interactive interface module, the psychoanalysis module is used for receiving voice data and image data of the user in multiple rounds of dialogues and analyzing keywords in the voice data and the image data and an emotional state of the user; the psychological counseling resource library is used for storing psychological counseling resources; the recommendation module is used for providing psychological consultation suggestions and resource recommendation for the user according to historical data, keywords and emotional states of the user; and the interactive interface module is used for displaying the recommended psychological counseling suggestions and resource recommendation. Based on the natural language processing technology and the emotion recognition technology, the method can improve the insight in the psychological state of the user in the consulting process, provides accurate psychological assessment in combination with voice, text and expression analysis, solves the limitation of the existing traditional consulting tool, improves the professional property of psychological consulting, and improves the consulting efficiency. And the working efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of psychological support technology, and in particular to a psychological counseling support system and psychological counseling method. Background Technology

[0002] Psychological counseling refers to the process of providing psychological assistance to those who have problems with psychological adaptation and seek solutions, using psychological methods. Clients describe, inquire about, and discuss their psychological discomfort or obstacles with the counselor through language, writing, or other communication media. Through joint discussion, the causes of psychological problems are identified in order to restore psychological balance and improve physical and mental health.

[0003] Currently, existing traditional counseling tools rely on manual recording and subjective judgment, lacking systematic analysis of emotional, vocal, and behavioral data, which may lead to the omission of important information. They also struggle to effectively organize, store, and retrieve text, audio, and video data generated during counseling, affecting long-term follow-up and research. The use of non-professional tools blurs the boundaries between counselors and clients, increasing ethical risks. Furthermore, their reliance on Western psychology without sufficient consideration of the Chinese language and cultural characteristics negatively impacts client acceptance and compliance. Therefore, there is an urgent need for a psychological counseling support system to overcome the limitations of existing traditional tools, thereby improving the professionalism of psychological counseling and increasing work efficiency. Summary of the Invention

[0004] This invention provides a psychological counseling assistance system, aiming to overcome the limitations of existing traditional counseling tools, improve the professionalism of psychological counseling, and increase work efficiency. This invention integrates a psychological analysis module, a psychological counseling resource library, a recommendation module, and an interactive interface module. The psychological analysis module receives voice and image data from users during multi-turn dialogues and analyzes the keywords and emotional states of the users in the voice and image data. The psychological counseling resource library stores psychological counseling resources. The recommendation module provides psychological counseling suggestions and resource recommendations to users based on their historical data, keywords, and emotional states. The interactive interface module displays the recommended psychological counseling suggestions and resource recommendations. Specifically, the system acquires voice and image data from users during multi-turn dialogues, extracts keywords from the voice data using natural language processing technology to obtain the user's current keywords, performs emotion recognition processing on the voice and / or image data using a preset emotion recognition model to obtain the user's current emotional state, acquires the user's historical data, matches psychological counseling suggestions and resource recommendations corresponding to the historical data, current keywords, and current emotional state in the psychological counseling resource library, and displays the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module. This invention, based on natural language processing and emotion recognition technologies, can improve the insight into the user's psychological state during the consultation process. By combining voice, text, and facial expression analysis, it provides accurate psychological assessments, overcomes the limitations of existing traditional consultation tools, and improves the professionalism of psychological counseling as well as work efficiency.

[0005] In a first aspect, embodiments of the present invention provide a psychological counseling assistance system, the system comprising a psychological analysis module, a psychological counseling resource library, a recommendation module, and an interactive interface module; the psychological analysis module is used to receive voice data and image data of users in multi-turn dialogues, and analyze the keywords and emotional states of users in the voice data and image data; the psychological counseling resource library is used to store psychological counseling resources; the recommendation module is used to provide psychological counseling suggestions and resource recommendations to users based on users' historical data, keywords, and emotional states; the interactive interface module is used to display the recommended psychological counseling suggestions and resource recommendations;

[0006] Specifically, the system acquires voice and image data from multi-turn dialogues, performs keyword extraction on the voice data using natural language processing technology to obtain the user's current keywords, performs emotion recognition processing on the voice and / or image data using a preset emotion recognition model to obtain the user's current emotional state, acquires the user's historical data, matches psychological counseling suggestions and resource recommendations corresponding to the historical data, current keywords, and current emotional state in the psychological counseling resource database, and displays the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module.

[0007] Optionally, the psychological analysis module includes a speech recognition submodule and an emotion recognition submodule. The speech recognition submodule is used to convert speech data into text data and extract keywords from the text data using natural language processing technology. The emotion recognition submodule is used to analyze the user's emotional state in the speech data and / or image data through an emotion recognition model.

[0008] Optionally, the emotion recognition model is obtained by training an untrained emotion recognition model using a training dataset, wherein the training dataset includes sample speech data and emotion annotation data corresponding to the sample speech data, sample image data and emotion annotation data corresponding to the sample image data.

[0009] Optionally, the psychological counseling resource database includes a localized case database, a localized case study database, and a localized advice and guidance database. The localized case database contains solutions and suggestions for different psychological problems and emotional states in the local context. The localized case study database contains successful localized psychological counseling cases and case analysis methods. The localized advice and guidance database provides specific suggestions and guidance for common localized psychological problems.

[0010] Optionally, the system further includes a compliance management module, which includes hierarchical access control, a boundary protection mechanism, a double-blind recording module, and an audit interface. The hierarchical access control is used to restrict communication outside of working hours. The boundary protection mechanism is used to detect communication requests outside of consultation hours and / or inappropriate topics. The double-blind recording module is used to store client identity information and consultation content separately, authorizing only psychological counselors to access complete data through biometric authentication. The audit interface is used to support third-party institutions in retrieving anonymized session records on demand, ensuring compliance review.

[0011] Optionally, the system further includes a language conversion module, which supports language conversion and performs real-time conversion based on the local language in the user's voice data.

[0012] Secondly, embodiments of the present invention also provide a psychological counseling method, which is applied to the psychological counseling support system as described in the embodiments of the present invention, the method comprising:

[0013] Acquire voice and image data from multi-turn dialogues;

[0014] The user's current keywords are obtained by extracting keywords from the voice data using natural language processing technology.

[0015] The user's current emotional state is obtained by performing emotion recognition processing on the voice data and / or the image data using a preset emotion recognition model.

[0016] The system acquires the user's historical data, matches psychological counseling suggestions and resource recommendations corresponding to the historical data, the current keywords, and the current emotional state in the psychological counseling resource library, and displays the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module.

[0017] Optionally, the step of extracting keywords from the voice data using natural language processing technology to obtain the user's current keywords includes:

[0018] The speech data is processed to convert speech to text, resulting in text data;

[0019] The text data is processed using natural language processing techniques to extract keywords, thereby obtaining the user's current keywords.

[0020] Optionally, before performing emotion recognition processing on the voice data and / or image data using a preset emotion recognition model to obtain the user's current emotional state, the method further includes:

[0021] Obtain a training data set and an untrained emotion recognition model. The training data set includes sample speech data and emotion annotation data corresponding to the sample speech data, sample image data and emotion annotation data corresponding to the sample image data.

[0022] The untrained emotion recognition model is trained using the training dataset, and once training is complete, a preset emotion recognition model is obtained.

[0023] Optionally, the step of performing emotion recognition processing on the voice data and / or the image data using a preset emotion recognition model to obtain the user's current emotional state includes:

[0024] The speech data is processed by a preset emotion recognition model to obtain the speech emotion recognition result.

[0025] And / or, perform image emotion recognition processing on the image data to obtain image emotion recognition results;

[0026] Based on the voice emotion recognition results and the image emotion recognition results, the current emotional state of the target user is obtained.

[0027] In this embodiment of the invention, a psychological analysis module, a psychological counseling resource library, a recommendation module, and an interactive interface module are integrated. The psychological analysis module receives voice and image data from users in multi-turn dialogues and analyzes the keywords and emotional states of the users in the voice and image data. The psychological counseling resource library stores psychological counseling resources. The recommendation module provides users with psychological counseling suggestions and resource recommendations based on their historical data, keywords, and emotional states. The interactive interface module displays the recommended psychological counseling suggestions and resource recommendations. Specifically, the system acquires voice and image data from users in multi-turn dialogues, extracts keywords from the voice data using natural language processing technology to obtain the user's current keywords, performs emotion recognition processing on the voice data and / or image data using a preset emotion recognition model to obtain the user's current emotional state, acquires the user's historical data, matches psychological counseling suggestions and resource recommendations corresponding to the historical data, current keywords, and current emotional state in the psychological counseling resource library, and displays the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module. This invention, based on natural language processing and emotion recognition technologies, can improve the insight into the user's psychological state during the consultation process. By combining voice, text, and facial expression analysis, it provides accurate psychological assessments, overcomes the limitations of existing traditional consultation tools, and improves the professionalism of psychological counseling as well as work efficiency. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a structural diagram of a psychological counseling assistance system provided in an embodiment of the present invention.

[0030] Figure 2 This is a flowchart of the psychological counseling method provided in the embodiments of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] like Figure 1 As shown, Figure 1 This is a structural diagram of a psychological counseling assistance system provided in an embodiment of the present invention. The psychological counseling assistance system includes a psychological analysis module, a psychological counseling resource library, a recommendation module, and an interactive interface module. The psychological analysis module receives voice and image data from users during multi-turn dialogues and analyzes the keywords and emotional states of the users in the voice and image data. The psychological counseling resource library stores psychological counseling resources. The recommendation module provides psychological counseling suggestions and resource recommendations to users based on their historical data, keywords, and emotional states. The interactive interface module displays the recommended psychological counseling suggestions and resource recommendations. Specifically, the system acquires voice and image data from users during multi-turn dialogues, extracts keywords from the voice data using natural language processing technology to obtain the user's current keywords, performs emotion recognition processing on the voice and / or image data using a preset emotion recognition model to obtain the user's current emotional state, acquires the user's historical data, matches psychological counseling suggestions and resource recommendations corresponding to the historical data, current keywords, and current emotional state in the psychological counseling resource library, and displays the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module.

[0033] In this embodiment of the invention, the above-mentioned psychological counseling assistance system is used in the online counseling process of psychological counselors.

[0034] The aforementioned psychological analysis module can analyze keywords and emotional states in users' speech data in real time using natural language processing (NLP) and computer vision technologies. NLP aims to enable computers to understand, interpret, and generate human language, thereby achieving effective human-computer communication. Keyword extraction can employ an NLP framework based on fine-tuning of the BERT model, with a training corpus of 100,000 anonymized psychological counseling dialogues (accuracy >90%). The emotion recognition model integrates speech (LSTM network) and image (ResNet50) features, optimized through a cross-entropy loss function. The training set includes 50,000 labeled speech samples and 30,000 emotion images (labeled according to Ekman's six emotion categories).

[0035] The aforementioned computer vision technology simulates biological vision using computers and related equipment. It processes acquired images or videos to obtain three-dimensional information about the corresponding scene. Specifically, computer vision technology uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments for detection.

[0036] The aforementioned emotional states can include happiness, sadness, anger, etc.

[0037] The aforementioned psychological counseling resource database is used to store and manage various psychological counseling resources. These resources aim to help counselors and users better understand and cope with mental health issues, providing professional advice and support.

[0038] The recommendation module employs pre-set scales and psychological approaches to generate a psychological state report based on real-time analysis of keywords, emotional states, and the user's historical data. It then combines this with psychological theories to generate diagnostic suggestions and provide relevant psychological counseling techniques and personalized intervention recommendations. The psychological states described reflect the user's emotions and mental state. The psychological theories mentioned can include cognitive behavioral therapy, psychodynamics, and other psychological theories. The psychological counseling techniques include cognitive restructuring techniques and relaxation training guidance. For example, if a user uses keywords related to anxiety and exhibits an anxious emotional state in a conversation, the recommendation module can recommend articles, videos, or online courses on how to alleviate anxiety based on the user's historical data, keywords, and emotional state.

[0039] The aforementioned pre-set scales are scales pre-configured by the system and can be scales such as the Symptom Checklist-90 (SCL-90) or the Patient Health Questionnaire-9 (PHQ-9). These scales are used to assess different mental health issues. The aforementioned schools of psychology can be understood as different theoretical systems and methodologies that have developed during the development of psychology, such as CBT and mindfulness therapy.

[0040] The resources mentioned above could include Confucian cultural and psychological education videos, and traditional Chinese medicine for emotional regulation.

[0041] The keyword extraction process described above can be understood as converting speech data into text data and identifying keywords within the text data. These keywords are words extracted from the text data that reflect its theme and core content; they can be keywords related to emotions, problem descriptions, goals, expectations, etc.

[0042] The aforementioned preset emotion recognition model can be an emotion recognition model built based on deep learning or machine learning, such as a Long Short-Term Memory Network (LSTM) or a Gated Recurrent Unit (GRU). The preset emotion recognition model can process speech and image data to identify and analyze the user's emotional state.

[0043] The aforementioned emotion recognition processing can be understood as the process of analyzing a user's voice data and / or image data to identify the emotional state expressed in the voice data and / or the emotional state expressed in the image data.

[0044] The aforementioned user historical data can be understood as users' historical consultation data, feedback data, behavioral data, etc. This historical data allows for a better understanding of user needs and behavioral habits.

[0045] The aforementioned interactive interface module also provides an immersive waiting room experience across devices. For example, users can experience bamboo forest meditation or ink painting interaction through VR devices, which helps enhance their psychological preparation before psychological counseling.

[0046] In this embodiment of the invention, the present invention, based on natural language processing technology and emotion recognition technology, can monitor conversation content in real time, automatically extract core questions, improve the insight into the user's psychological state during the consultation process, and provide accurate psychological assessment by combining voice and image emotion state analysis, thus overcoming the limitations of existing traditional consultation tools, improving the professionalism of psychological counseling, and increasing work efficiency.

[0047] In this embodiment, a psychological analysis module, a psychological counseling resource library, a recommendation module, and an interactive interface module are integrated. The psychological analysis module receives voice and image data from users in multi-turn dialogues and analyzes the keywords and emotional states of the users in the voice and image data. The psychological counseling resource library stores psychological counseling resources. The recommendation module provides users with psychological counseling suggestions and resource recommendations based on their historical data, keywords, and emotional states. The interactive interface module displays the recommended psychological counseling suggestions and resource recommendations. Specifically, voice and image data from multi-turn dialogues are acquired, and natural language processing technology is used to extract keywords from the voice data to obtain the user's current keywords. A preset emotion recognition model is used to perform emotion recognition processing on the voice data and / or image data to obtain the user's current emotional state. The user's historical data is acquired, and psychological counseling suggestions and resource recommendations corresponding to the historical data, current keywords, and current emotional state are matched in the psychological counseling resource library. The corresponding psychological counseling suggestions and resource recommendations are then displayed through the interactive interface module. This invention, based on natural language processing and emotion recognition technologies, can improve the insight into the user's psychological state during the consultation process. By combining voice, text, and facial expression analysis, it provides accurate psychological assessments, overcomes the limitations of existing traditional consultation tools, and improves the professionalism of psychological counseling as well as work efficiency.

[0048] Optionally, the psychoanalysis module includes a speech recognition submodule and an emotion recognition submodule.

[0049] In this embodiment of the invention, the aforementioned speech recognition submodule is used to convert speech data into text data and apply natural language processing (NLP) technology to extract keywords from the text data. The aforementioned NLP technology aims to enable computers to understand, interpret, and generate human language. The goal of NLP technology is to enable computers to understand, interpret, and generate human language, thereby achieving effective communication between humans and machines.

[0050] Furthermore, a keyword extraction model based on natural language processing technology can be used for keyword extraction. This keyword extraction model is trained on a de-identified database of real psychological counseling data, ensuring the accuracy of the data analysis.

[0051] The aforementioned emotion recognition submodule is used to analyze the user's emotional state in voice data and / or image data using a multimodal emotion recognition model.

[0052] The aforementioned emotion recognition model can process voice and image data to identify and analyze the user's emotional state.

[0053] It should be noted that the psychoanalysis module can extract keywords from the dialogue at different stages of the psychological counseling process. This module can extract key information and data at different stages, helping the system to gain a more comprehensive understanding of the counseling process and providing a basis for intervention and recommendations. The aforementioned psychological counseling process includes initial assessment, psychological support planning, intervention trial, progress tracking, and case closure.

[0054] Optionally, the emotion recognition model is obtained by training an untrained emotion recognition model using a training dataset.

[0055] In this embodiment of the invention, the training dataset includes sample speech data and corresponding emotion annotation data, sample image data and corresponding emotion annotation data.

[0056] The aforementioned untrained emotion recognition model can be built based on machine learning or deep learning models, such as Long Short-Term Memory Network (LSTM), Gated Recurrent Unit (GRU), etc.

[0057] The aforementioned sample speech data can be speech data in different emotional states, including speech data in emotional states such as happiness, sadness, and anger.

[0058] The emotion annotation data corresponding to the above sample speech data can be understood as the emotion annotation performed on the sample speech data. The annotation data indicates the emotional state expressed by each speech segment.

[0059] The sample image data mentioned above can be image data in different emotional states, including image data of happiness, sadness, anger, etc.

[0060] The aforementioned sample image data and the corresponding emotion annotation data can be understood as emotion annotations performed on the image data, indicating the emotional state expressed by the image.

[0061] The labeled data mentioned above can be understood as adding structured labels to the raw data, enabling machine learning models to recognize and process both the raw and labeled data. Through the labeled data, the model can establish a mapping relationship between the input data and the correct output labels.

[0062] The training described above can be supervised training. Supervised training uses a set of data with known labels to train the model. By optimizing the model parameters, the model can predict the labels of new data or make decisions based on the characteristics of existing data.

[0063] During training, the minimum loss function can be used to adjust the model's parameters to minimize the difference between the model's output label and the input data.

[0064] The loss function described above measures the difference between the model's predictions and the actual results. Its purpose is to improve prediction accuracy by minimizing the loss function value through adjusting the model parameters. This loss function can be the mean squared error loss function, cross-entropy loss function, etc.

[0065] Optionally, the psychological counseling resource database includes a localized case database, a localized case study database, and a localized advice and guidance database.

[0066] In this embodiment of the invention, the aforementioned localized case library contains solutions and suggestions for different psychological problems and emotional states in the local context. The aforementioned localized case study library contains successful localized psychological counseling cases and case analysis methods. The aforementioned localized suggestion and guidance library provides specific suggestions and guidance for common localized psychological problems.

[0067] The aforementioned localized case library is based on a Chinese cultural context, including cases on family relationship mediation, interpersonal relationship management, and seasonal mood regulation. It should be noted that the psychological counseling resource library of this invention also includes domestic and international clinical trial data.

[0068] Optionally, the system also includes a compliance management module, which includes hierarchical access control, boundary protection mechanisms, a double-blind recording module, and an audit interface.

[0069] In this embodiment of the invention, the above-mentioned hierarchical access control is used to restrict communication outside of working hours to prevent psychological counselors from having private contact with users.

[0070] The aforementioned boundary protection mechanism is used to detect communication requests and / or inappropriate topics outside of consultation hours. It automatically detects communication requests between the therapist and user outside of consultation hours, or inappropriate topics such as private social invitations. When a therapist requests communication with a user outside of consultation hours, an alert is triggered and logged; or when a therapist and user discuss inappropriate topics, an alert is triggered and logged. The boundary protection mechanism can employ an NLP sensitive topic classification model (e.g., detecting keywords such as 'private social invitations' and 'home address'), and automatically generates logs and pushes them to the auditing interface after an alert is triggered.

[0071] The aforementioned double-blind recording module separates client identity information from consultation content for storage, authorizing only psychological counselors to access the complete data via biometric authentication. This biometric authentication can be understood as identity verification using physiological characteristics, such as fingerprint authentication. This separation of client identity information from consultation content enhances data security and privacy. The double-blind recording module's access to complete data via fingerprint biometric authentication complies with relevant laws and regulations.

[0072] The aforementioned auditing interface supports third-party organizations in retrieving anonymized session logs on demand, ensuring compliance audits. Through this interface, third-party organizations can access anonymized session logs as needed, ensuring the protection of users' personally identifiable information.

[0073] It should be noted that the compliance management module is used to follow ethical norms and ensure that the psychological counseling process complies with the "Ethical Code of Clinical and Counseling Psychology of the Chinese Psychological Society".

[0074] Optionally, the system also includes a language conversion module, which supports language conversion and performs real-time conversion based on the local language in the user's voice data.

[0075] In this embodiment of the invention, the language conversion module can perform real-time conversion based on the local language used in the user's voice data. The local language can be Cantonese, Minnan (Hokkien), or other minority languages. The language conversion module supports real-time conversion of multiple dialects, including Cantonese and Minnan, and employs an end-to-end dialect speech recognition model (based on the Wav2Vec 2.0 framework), with training data covering various dialect dialogue corpora.

[0076] In this embodiment of the invention, the invention supports compatibility with Mandarin and local languages, reducing the impact of language barriers on the effectiveness of consultation, so as to better understand and respond to user needs.

[0077] It should be noted that this invention employs an automated workflow engine, which generates electronic invoices with a single click, supporting QR code payments and electronic signatures. The automated workflow engine also enables intelligent schedule synchronization, cross-device conflict detection and alerts, and automatic adjustment of appointment priorities based on user urgency. Furthermore, it performs intelligent document categorization, automatically linking session records, assessment reports, and legal documents. The automated workflow engine includes a built-in legal document template library, such as templates for informed consent forms and confidentiality agreements, supporting one-click generation and binding of electronic signatures. It supports multi-platform payment aggregation, integrating WeChat Pay, Alipay, and UnionPay interfaces to automatically reconcile accounts and generate tax compliance reports. This invention, utilizing an automated workflow engine, integrates electronic signatures, QR code payments, intelligent schedule management, and document categorization functions, covering the entire process of administrative affairs automation before, during, and after consultations. This reduces administrative operation time and improves service standardization.

[0078] It should be noted that this invention can automatically trigger the emergency contact, Binggang, to push an intervention protocol when high-risk signals are detected, thus improving the user experience. This invention also provides follow-up tracking of post-psychological counseling exercises and visual feedback after counseling.

[0079] In one possible embodiment, during psychological counseling, the counselor initiates video counseling with the user during working hours. This invention collects the user's voice and image data in real time, extracts keywords and the user's emotional state, and automatically generates an analysis report after the session ends, which is then pushed to the client.

[0080] In another possible embodiment, during psychological counseling, after the user completes a self-assessment questionnaire about their mental state, this invention can generate a preliminary mental health assessment report based on the user's input text or voice, monitor the conversation content in real time, automatically extract core questions, and generate structured counseling notes. This invention can automatically monitor sensitive dialogues to ensure that the counseling process meets professional standards. After the session ends, this invention automatically generates a personalized counseling summary and provides follow-up mental health suggestions. This invention can improve the accuracy of psychological counseling and reduce human judgment bias.

[0081] like Figure 2 As shown, Figure 2 This is a flowchart of a psychological counseling method provided in an embodiment of the present invention, which includes the following steps:

[0082] 201. Obtain user voice and image data during multi-turn dialogues.

[0083] In this embodiment of the invention, the above-described psychological counseling method can be applied to a psychological counseling assistance system. The system includes a psychological analysis module, a psychological counseling resource library, a recommendation module, and an interactive interface module. The psychological analysis module receives voice and image data from users during multi-turn dialogues and analyzes the keywords and emotional states of the users within the voice and image data. The psychological counseling resource library stores psychological counseling resources. The recommendation module provides users with psychological counseling suggestions and resource recommendations based on their historical data, keywords, and emotional states. The interactive interface module displays the recommended psychological counseling suggestions and resource recommendations.

[0084] The aforementioned multi-round dialogues can be understood as the multi-stage dialogues between the therapist and the client during the psychological counseling process. Specifically, these dialogues can be based on the various stages of the psychological counseling process, which includes initial assessment, psychological support planning, intervention implementation, progress tracking, and case closure.

[0085] 202. Use natural language processing technology to extract keywords from the voice data to obtain the user's current keywords.

[0086] In this embodiment of the invention, the Natural Language Processing (NLP) technology aims to enable computers to understand, interpret, and generate human language. The goal of NLP is to enable computers to understand, interpret, and generate human language, thereby achieving effective communication between humans and machines. Specifically, a keyword extraction model built based on NLP technology can be used for keyword extraction. This keyword extraction model is trained on a de-identified database of real psychological counseling data, ensuring the accuracy of data analysis.

[0087] The keyword extraction process described above can be understood as the process of converting speech data into text data and identifying keywords in the text data.

[0088] The keywords mentioned above are words extracted from text data that reflect its theme and core content. They can be keywords related to emotions, keywords describing problems, keywords related to goals and expectations, etc.

[0089] 203. Perform emotion recognition processing on voice data and / or image data using a preset emotion recognition model to obtain the user's current emotional state.

[0090] In this embodiment of the invention, the aforementioned preset emotion recognition model can be an emotion recognition model built based on deep learning or machine learning, such as a Long Short-Term Memory Network (LSTM), a Gated Recurrent Unit (GRU), etc. The preset emotion recognition model can process speech data and image data to identify and analyze the user's emotional state.

[0091] The aforementioned emotion recognition processing can be understood as the process of analyzing a user's voice data and / or image data to identify the emotional state expressed in the voice data and / or the emotional state expressed in the image data.

[0092] The aforementioned emotional states can include happiness, sadness, anger, etc.

[0093] In one possible embodiment, the above-mentioned emotion recognition processing can be understood as the process of analyzing a user's voice data, image data, and physiological data to identify the emotional state expressed in the voice data, the emotional state expressed in the image data, and the emotional state expressed in the physiological data.

[0094] The physiological stress index (0-100) was generated by analyzing HRV time-series data using a GRU network.

[0095] It can combine speech (LSTM) and image (ResNet-50) emotion recognition results and dynamically allocate weights through an attention mechanism.

[0096] Final emotional state E final The formula is as follows:

[0097] E final =α·E voice +β·E image +γ·E bio

[0098] Among them, E voice E represents the emotional state expressed in the speech data. image E represents the emotional state expressed in the image data. bio The weights α, β, and γ can be adaptively adjusted based on historical data to represent the emotional state expressed in physiological data, for example, α = 0.4, β = 0.3, and γ = 0.3.

[0099] It should be noted that the user's current emotional state can be obtained by performing emotion recognition processing on voice data using a preset emotion recognition model; the user's current emotional state can also be obtained by performing emotion recognition processing on image data using a preset emotion recognition model; or the user's current emotional state can be obtained by performing emotion recognition processing on both voice data and image data using a preset emotion recognition model.

[0100] 204. Obtain the user's historical data, match the psychological counseling resource library with the historical data, current keywords, and current emotional state to provide psychological counseling suggestions and resource recommendations, and display the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module.

[0101] In this embodiment of the invention, the aforementioned user's historical data includes historical consultation data, feedback data, behavioral data, etc. User historical data allows for a better understanding of user needs and behavioral habits.

[0102] The aforementioned psychological counseling resource database is used to store and manage various psychological counseling resources. These resources aim to help counselors and users better understand and cope with mental health issues, providing professional advice and support.

[0103] The aforementioned interactive interface module is used to display recommended psychological counseling suggestions and resource recommendations.

[0104] In one possible implementation, for example, if a user uses keywords related to anxiety and has an anxious emotional state in a conversation, articles, videos, or online courses on how to alleviate anxiety can be recommended to the user based on the user's historical data, keywords, and emotional state, and these articles, videos, or online courses can be displayed through an interactive interface module.

[0105] In this embodiment, voice and image data from multi-turn dialogues are acquired. Natural language processing (NLP) is used to extract keywords from the voice data to obtain the user's current keywords. A preset emotion recognition model is then used to perform emotion recognition processing on the voice and / or image data to determine the user's current emotional state. The user's historical data is also acquired. Psychological counseling suggestions and resource recommendations corresponding to the historical data, current keywords, and current emotional state are matched against a psychological counseling resource database. These suggestions and recommendations are then displayed through an interactive interface module. This invention, based on NLP and an emotion recognition model, can analyze voice and image data in real time, improving the insight into the user's psychological state during psychological counseling. Furthermore, by combining voice, text, and facial expression analysis, it provides accurate psychological assessments.

[0106] It is understood that in the specific implementation of this application, data such as voice data, image data, knowledge data, and user data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use, and processing of related data, as well as the training, deployment, and invocation of algorithm models, must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0107] Optionally, in the step of extracting keywords from the voice data using natural language processing technology to obtain the user's current keywords, the voice data can be converted to text to obtain text data; and the text data can then be processed using natural language processing technology to extract keywords to obtain the user's current keywords.

[0108] In this embodiment of the invention, the above-mentioned speech-to-text processing can be understood as the process of converting speech data into text data.

[0109] The aforementioned natural language processing (NLP) technology aims to enable computers to understand, interpret, and generate human language, thereby achieving effective communication between humans and machines. Specifically, a keyword extraction model built upon NLP technology can be used for keyword extraction. This keyword extraction model is trained on a de-identified database of real psychological counseling data, ensuring the accuracy of data analysis.

[0110] In one possible implementation, user privacy can be protected during data sharing and model training by adding Laplace noise (ε = 0.1) to the consultation text to ensure that individual records are not traceable. The magnitude of model parameter updates is limited during federated learning (threshold = 1.0). The total privacy budget ε_total = 2.0 is allocated over 10 training rounds (ε = 0.2 per round). (ε,δ)-differential privacy (δ = 1e-5) is satisfied, with data availability loss <5%.

[0111] The keyword extraction process described above can be understood as the process of identifying keywords in text data.

[0112] The aforementioned current keywords are words extracted from text data that reflect its theme and core content. These can be keywords related to emotions, keywords describing problems, keywords related to goals and expectations, etc.

[0113] It should be noted that this invention obtains text data by converting speech data to text, and then extracts keywords from the text data using natural language processing technology to obtain the user's current keywords. Natural language processing technology can analyze and process speech data more effectively.

[0114] Optionally, before obtaining the user's current emotional state by performing emotion recognition processing on the voice data and / or image data using a preset emotion recognition model, a data training set and an untrained emotion recognition model can be obtained; the untrained emotion recognition model can be trained using the training dataset, and after training is completed, the preset emotion recognition model is obtained.

[0115] In this embodiment of the invention, the above-mentioned data training set includes sample speech data and emotion annotation data corresponding to the sample speech data, sample image data and emotion annotation data corresponding to the sample image data.

[0116] The aforementioned sample speech data can be speech data in different emotional states, including speech data in emotional states such as happiness, sadness, and anger.

[0117] The emotion annotation data corresponding to the above sample speech data can be understood as the emotion annotation performed on the sample speech data. The annotation data indicates the emotional state expressed by each speech segment.

[0118] The sample image data mentioned above can be image data in different emotional states, including image data of happiness, sadness, anger, etc.

[0119] The aforementioned sample image data and the corresponding emotion annotation data can be understood as emotion annotations performed on the image data, indicating the emotional state expressed by the image.

[0120] The labeled data mentioned above can be understood as adding structured labels to the raw data, enabling machine learning models to recognize and process both the raw and labeled data. Through the labeled data, the model can establish a mapping relationship between the input data and the correct output labels.

[0121] The aforementioned untrained emotion recognition model can be built based on machine learning or deep learning models, such as Long Short-Term Memory Network (LSTM), Gated Recurrent Unit (GRU), etc.

[0122] The training described above can be supervised training. Supervised training uses a set of data with known labels to train the model. By optimizing the model parameters, the model can predict the labels of new data or make decisions based on the characteristics of existing data.

[0123] The aforementioned preset emotion recognition model can process voice and image data to identify and analyze the user's emotional state.

[0124] It should be noted that the untrained emotion recognition model is trained using the training dataset. During the training process, the parameters of the model can be adjusted using the minimum loss function to minimize the difference between the output label and the input data. Once training is complete, the preset emotion recognition model is obtained.

[0125] The loss function described above measures the difference between the model's predictions and the actual results. Its purpose is to improve prediction accuracy by minimizing the loss function value through adjusting the model parameters. This loss function can be the mean squared error loss function, cross-entropy loss function, etc.

[0126] Optionally, in the step of performing emotion recognition processing on voice data and / or image data through a preset emotion recognition model to obtain the user's current emotional state, the preset emotion recognition model can be used to perform voice emotion recognition processing on the voice data to obtain a voice emotion recognition result; and / or, image emotion recognition processing can be performed on the image data to obtain an image emotion recognition result; based on the voice emotion recognition result and the image emotion recognition result, the current emotional state of the target user can be obtained.

[0127] In this embodiment of the invention, the aforementioned preset emotion recognition model can process voice data and image data to identify and analyze the user's emotional state.

[0128] The above-mentioned speech emotion recognition processing can be understood as the process of analyzing speech data to identify the speaker's emotional state.

[0129] The above voice emotion recognition results are obtained by performing voice emotion recognition on voice data. The above voice emotion recognition results can be happy emotions, sad emotions, etc.

[0130] The above image emotion recognition processing can be understood as the process of analyzing image data to identify the emotional state of a person's face in the image.

[0131] The above image emotion recognition results are obtained by performing image emotion recognition on image data. The above image emotion recognition results can be happy, sad, etc.

[0132] In one possible embodiment, speech data can be processed by a preset emotion recognition model to obtain speech emotion recognition results; image data can be processed by the preset emotion recognition results to obtain image emotion recognition results; and speech data and image data can be processed by a preset emotion recognition model to obtain speech emotion recognition results and image emotion recognition results.

[0133] During psychological counseling, a pre-trained cross-modal emotion recognition model can simultaneously analyze the user's speech and image data. For example, when a user describes "recently experiencing insomnia," the speech model (based on a self-supervised learning Wav2Vec2.0 architecture) captures the tremor at the end of the voice and a sudden drop in speech rate. Combined with the low-frequency energy attenuation features in the spectrogram, it identifies the emotion as "anxiety" (83% confidence). At the same time, the image model (integrating Vision Transformer and 3D convolutional network) detects the user's unconscious lip biting and neck muscle tension micro-movement sequences, identifying them as a state of "stress accumulation." To overcome the limitations of a single modality, the system introduces a multimodal alignment algorithm based on causal reasoning to dynamically analyze the temporal synchronization of speech and facial expressions. If a user claims "I'm fine" but is accompanied by a brief closing of the eyes and a twitching of the corner of the mouth, the model automatically marks it as "potential emotion masking" and triggers deep physiological signal verification (such as non-contact monitoring of heart rate variability and respiratory rate via millimeter-wave radar). In addition, the system has a built-in cultural sensitivity compensation mechanism. For example, it automatically loads a "context enhancement module" to address the common emotional suppression characteristics of East Asian users, and increases the recognition weight of "latent anxiety" by combining the dialogue context (such as when the user mentions "family expectations").

[0134] When multimodal results conflict, the system invokes a meta-decision network optimized through transfer learning. It references the user's historical emotional expression patterns during consultations (e.g., a user's habit of remaining silent and bowing their head when angry) and incorporates real-time environmental sensor data (e.g., room lighting intensity and background noise levels) for adaptive calibration. This results in an emotional heatmap with a reliable range, which is then overlaid with visual cues (e.g., floating red "high-stress zone" markers) on the counselor via augmented reality glasses. To address extreme scenarios, the system integrates a generative adversarial network (GAN) to simulate rare emotional combinations (e.g., a "sad-numb" mixed state). Synthetic data enhances the model's ability to generalize to complex psychological states, ensuring high recognition accuracy even in borderline cases encountered in real-world consultations (e.g., the dissociative reactions of patients with PTSD).

[0135] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0136] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A psychological counseling support system, characterized in that, The system includes a psychological analysis module, a psychological counseling resource library, a recommendation module, and an interactive interface module. The psychological analysis module is used to receive users' voice data and image data in multi-turn dialogues, and analyze the keywords and users' emotional states in the voice data and image data. The psychological counseling resource library is used to store psychological counseling resources; the recommendation module is used to provide psychological counseling suggestions and resource recommendations to the user based on the user's historical data, keywords, and emotional state; the interactive interface module is used to display the recommended psychological counseling suggestions and resource recommendations. Specifically, the process involves acquiring the user's voice and image data during multi-turn dialogues, extracting keywords from the voice data using natural language processing technology to obtain the user's current keywords, and performing emotion recognition processing on the voice data and / or image data using a preset emotion recognition model to obtain the user's current emotional state. The system acquires the user's historical data, matches psychological counseling suggestions and resource recommendations corresponding to the historical data, the current keywords, and the current emotional state in the psychological counseling resource library, and displays the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module.

2. The psychological counseling support system as described in claim 1, characterized in that, The psychological analysis module includes a speech recognition submodule and an emotion recognition submodule. The speech recognition submodule is used to convert speech data into text data and extract keywords from the text data using natural language processing technology. The emotion recognition submodule is used to analyze the user's emotional state in the speech data and / or image data through an emotion recognition model.

3. The psychological counseling support system as described in claim 2, characterized in that, The emotion recognition model is obtained by training an untrained emotion recognition model using a training dataset. The training dataset includes sample speech data and emotion annotation data corresponding to the sample speech data, sample image data and emotion annotation data corresponding to the sample image data.

4. The psychological counseling support system as described in claim 1, characterized in that, The psychological counseling resource database includes a localized case database, a localized case study database, and a localized advice and guidance database. The localized case database contains solutions and suggestions for different psychological problems and emotional states in the local context. The localized case study database contains successful localized psychological counseling cases and case analysis methods. The localized advice and guidance database provides specific suggestions and guidance for common localized psychological problems.

5. The psychological counseling support system as described in claim 1, characterized in that, The system also includes a compliance management module, which comprises hierarchical access control, a boundary protection mechanism, a double-blind recording module, and an audit interface. The hierarchical access control restricts communication outside of working hours. The boundary protection mechanism detects communication requests outside of consultation hours and / or inappropriate topics. The double-blind recording module stores client identity information separately from consultation content, authorizing only psychological counselors to access complete data via biometric authentication. The audit interface supports third-party institutions in retrieving anonymized session records on demand, ensuring compliance review.

6. The psychological counseling support system as described in claim 1, characterized in that, The system also includes a language conversion module, which supports language conversion and performs real-time conversion based on the local language in the user's voice data.

7. A psychological counseling method, characterized in that, The psychological counseling method is applied to the psychological counseling support system as described in any one of claims 1-6, wherein the method comprises: Acquire user voice and image data during multi-turn conversations; The user's current keywords are obtained by extracting keywords from the voice data using natural language processing technology. The user's current emotional state is obtained by performing emotion recognition processing on the voice data and / or the image data using a preset emotion recognition model. The system acquires the user's historical data, matches psychological counseling suggestions and resource recommendations corresponding to the historical data, the current keywords, and the current emotional state in the psychological counseling resource library, and displays the corresponding psychological counseling suggestions and resource recommendations through the interactive interface module.

8. The method as described in claim 7, characterized in that, The step of extracting keywords from the voice data using natural language processing technology to obtain the user's current keywords includes: The speech data is processed to convert speech to text, resulting in text data; The text data is processed using natural language processing techniques to extract keywords, thereby obtaining the user's current keywords.

9. The method as described in claim 8, characterized in that, Before performing emotion recognition processing on the voice data and / or image data using a preset emotion recognition model to obtain the user's current emotional state, the method further includes: Obtain a training data set and an untrained emotion recognition model. The training data set includes sample speech data and emotion annotation data corresponding to the sample speech data, sample image data and emotion annotation data corresponding to the sample image data. The untrained emotion recognition model is trained using the training dataset, and once training is complete, a preset emotion recognition model is obtained.

10. The method as described in claim 9, characterized in that, The step of performing emotion recognition processing on the voice data and / or image data using a preset emotion recognition model to obtain the user's current emotional state includes: The speech data is processed by a preset emotion recognition model to obtain the speech emotion recognition result. And / or, perform image emotion recognition processing on the image data to obtain image emotion recognition results; Based on the voice emotion recognition results and the image emotion recognition results, the current emotional state of the target user is obtained.

Images