Psychological consultation auxiliary system and method based on user feature analysis

Abstract

The invention discloses a psychological counseling assisting system and method based on user feature analysis, and the method comprises a data collection module which is used for responding to an online psychological counseling starting signal of a target user, acquiring a facial image of a target user acquired by a camera device, acquiring voice data of the target user acquired by a sound receiving device, and acquiring physiological signal data of the target user acquired by intelligent monitoring equipment worn by the target user; the feature analysis module is used for extracting facial features, sound features and physiological features and analyzing emotional state information of the target user in real time according to the facial features, the sound features and the physiological features; the feedback module is used for generating emotion feedback information according to the emotion state information of the target user and sending the emotion feedback information to the display equipment; and the display device is used for displaying the emotion feedback information so that the psychological consultant can continue to provide psychological consultation service for the target user after reference.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a psychological counseling assistance system and method based on user feature analysis. Background Art

[0002] In today's fast-paced and high-pressure social environment, mental health issues have become increasingly prominent and have become one of the important factors affecting people's quality of life. Traditional psychological counseling methods mainly rely on face-to-face conversations. Psychological counselors evaluate the client's mental state and develop corresponding counseling strategies by observing the client's verbal expression, non-verbal behavior (such as facial expressions, body language), and voice characteristics. However, this method is limited by the counselor's subjective judgment and experience level, and it is difficult to fully and objectively capture and quantify the client's emotional changes. Especially when faced with complex or hidden emotional expressions, it is easy to make misjudgments or omissions.

[0003] In addition, with the rapid development of information technology, especially the widespread application of technologies such as big data and artificial intelligence, new opportunities and challenges have been brought to the field of psychological counseling. The traditional counseling model has gradually shifted towards digitalization and intelligence in order to achieve more accurate and efficient psychological counseling services. However, most of the psychological counseling assistance systems on the market currently focus on self-assessment in the form of text or questionnaires, lacking the ability to monitor and analyze the emotional state of visitors in real time and in multiple dimensions, and unable to provide immediate and comprehensive reference information for psychological counselors. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a psychological counseling assistance system and method based on user feature analysis. Through comprehensive analysis of multi-source data, the system can more comprehensively capture changes in the user's emotional state and provide more accurate emotional feedback to psychological counselors, thereby helping counselors to develop more personalized counseling plans.

[0005] According to one aspect of the present application, a psychological consultation assistance system based on user feature analysis is provided, comprising:

[0006] A data acquisition module, for responding to a target user's online psychological counseling start signal, acquiring a facial image of the target user acquired by a camera device, acquiring voice data of the target user acquired by a sound collecting device, and acquiring physiological signal data of the target user acquired by an intelligent monitoring device worn by the target user;

[0007] A feature analysis module, used to extract facial features corresponding to the facial image, sound features corresponding to the voice data, and physiological features corresponding to the physiological signal data, and analyze the emotional state information of the target user in real time based on the facial features, the sound features, and the physiological features;

[0008] A feedback module, used to generate emotional feedback information according to the emotional state information of the target user, and send the emotional feedback information to a display device;

[0009] The display device is used to display the emotional feedback information for reference by the psychological counselor so as to continue to provide psychological counseling services to the target user.

[0010] In the embodiment of the present application, optionally, the feature analysis module is further used to:

[0011] The facial features, the voice features and the physiological features are fused to obtain a user fusion feature, and the emotional state of the target user is analyzed in real time using the user fusion feature to obtain first emotional state information;

[0012] Performing real-time emotional state analysis on the facial features, the voice features, and the physiological features respectively, and fusing the facial feature emotional analysis results, the voice feature emotional analysis results, and the physiological feature emotional analysis results to obtain second emotional state information;

[0013] The emotional state information of the target user is determined according to the first emotional state information and the second emotional state information.

[0014] In the embodiment of the present application, optionally, the facial features include position features and morphological features of facial expression feature points, the voice features include pitch features, speech speed features and volume features, and the physiological features include heart rate features, blood pressure features and skin electrical response features;

[0015] The feature analysis module is also used for:

[0016] Respectively comparing the position features of the facial expression feature points, the morphological features of the facial expression feature points, the tone features, the speech speed features, the volume features, the heart rate features, the blood pressure features, and the skin electrical response features with the features of the previous time period corresponding to the respective feature types, and determining the feature change amplitude corresponding to each feature type;

[0017] Determine the facial feature change amplitude according to the position feature change amplitude of the facial expression feature point and the morphological feature change amplitude of the facial expression point, determine the sound feature change amplitude according to the pitch feature change amplitude, the speech speed feature change amplitude and the volume feature change amplitude, and determine the physiological feature change amplitude according to the heart rate feature change amplitude, the blood pressure feature change amplitude and the skin electrical response feature change amplitude;

[0018] Determining facial feature weights, voice feature weights, and physiological feature weights according to the facial feature change range, the voice feature change range, and the physiological feature change range;

[0019] According to the facial feature weight, the voice feature weight and the physiological feature weight, the facial feature emotion analysis result, the voice feature emotion analysis result and the physiological feature emotion analysis result are fused to obtain the second emotion state information.

[0020] In the embodiment of the present application, optionally, the feature analysis module is further used to:

[0021] Comparing and analyzing the first emotional state information and the second emotional state information, and generating emotional state recognition difference information of the target user according to the difference between the second emotional state information and the first emotional state information;

[0022] If the emotional state recognition difference information is within a preset threshold, determining the average of the first emotional state information and the second emotional state information as the emotional state information;

[0023] If the emotional state recognition difference information is not within the preset threshold, the difference between the emotional state recognition difference information and the preset threshold is calculated, the ratio of the difference to the preset threshold is calculated, the difference between 0.5 and the ratio is used to determine the fusion weight of the first emotional state information, the sum of 0.5 and the ratio is determined as the fusion weight of the second emotional state information, and the first emotional state information and the second emotional state information are weightedly fused to determine the emotional state information.

[0024] In the embodiment of the present application, optionally, the feature analysis module is further used to:

[0025] The feedback module is further used to generate emotion change information according to the emotion state information and the emotion state information of the previous period corresponding to the emotion state information, and send the emotion change information to the display device;

[0026] The display device is also used to display the emotion feedback information and the emotion change information.

[0027] In the embodiment of the present application, optionally,

[0028] The data acquisition module is also used to obtain the audio data recorded during the online psychological consultation process for the target user, and to convert the audio data into text to obtain the consultation dialogue data of the online psychological consultation;

[0029] The feedback module is also used to generate emotional fluctuation information of online psychological consultation based on the consultation dialogue data and the first emotional state information.

[0030] In the embodiment of the present application, optionally,

[0031] The feedback module is also used to divide the consultation conversation data into multiple conversation segment data according to different emotional state types based on the time correspondence between the first emotional state information and the consultation conversation data, and divide the conversation segment data into conversation segment data groups according to the emotional state types, analyze the key information that causes the change of the emotional state type of the target user based on the conversation segment data groups, and push it to the terminal device of the psychological counselor according to the key information and the corresponding emotional state type.

[0032] In the embodiment of the present application, optionally, it also includes:

[0033] A model training module, used to construct training samples according to the emotional state information and the physiological characteristics, and use the training samples to train the physiological characteristic emotion recognition model corresponding to the target user;

[0034] The emotion monitoring module is used to obtain the daily physiological signal data of the target user in daily life collected by the smart monitoring device worn by the target user, extract the daily physiological characteristics corresponding to the daily physiological signal data, and use the physiological characteristic emotion recognition model to identify the daily emotional state information of the target user according to the daily physiological characteristics.

[0035] In the embodiment of the present application, optionally,

[0036] The emotion monitoring module is also used to push a mental health resource link to the terminal device of the target user based on the daily emotion state information, and to push the daily emotion state information to the terminal device of the psychological counselor.

[0037] According to another aspect of the present application, a psychological counseling assistance method based on user feature analysis is provided, which is applied to the above-mentioned psychological counseling assistance system based on user feature analysis, and includes:

[0038] The data acquisition module responds to the target user's online psychological consultation start signal to acquire the target user's facial image acquired by a camera device, acquire the target user's voice data acquired by a sound collection device, and acquire the target user's physiological signal data acquired by a smart monitoring device worn by the target user;

[0039] The feature analysis module extracts facial features corresponding to the facial image, sound features corresponding to the voice data, and physiological features corresponding to the physiological signal data, and analyzes the emotional state information of the target user in real time based on the facial features, the sound features, and the physiological features;

[0040] The feedback module generates emotional feedback information according to the emotional state information of the target user, and sends the emotional feedback information to the display device;

[0041] The display device displays the emotional feedback information for reference by the psychological counselor who then continues to provide psychological counseling services to the target user.

[0042] In an embodiment of the present application, optionally, analyzing the target user's emotional state information in real time according to the facial features, the voice features, and the physiological features includes:

[0043] The feature analysis module fuses the facial features, the voice features and the physiological features to obtain a user fusion feature, and performs real-time analysis of the emotional state of the target user using the user fusion feature to obtain first emotional state information; performs real-time analysis of the emotional state of the facial features, the voice features and the physiological features respectively, and fuses the facial feature emotional analysis results, the voice feature emotional analysis results and the physiological feature emotional analysis results to obtain second emotional state information; determines the emotional state information of the target user based on the first emotional state information and the second emotional state information.

[0044] In the embodiment of the present application, optionally, the facial features include position features and morphological features of facial expression feature points, the voice features include pitch features, speech speed features and volume features, and the physiological features include heart rate features, blood pressure features and skin electrical response features;

[0045] The facial feature emotion analysis results, the voice feature emotion analysis results, and the physiological feature emotion analysis results are integrated to obtain the second emotion state information, including:

[0046] The feature analysis module compares the position features of the facial expression feature points, the morphological features of the facial expression feature points, the pitch features, the speech rate features, the volume features, the heart rate features, the blood pressure features and the skin galvanic response features with the features of the previous time period corresponding to the respective feature types to determine the feature change amplitude corresponding to each feature type; determines the facial feature change amplitude according to the position feature change amplitude of the facial expression feature points and the morphological feature change amplitude of the facial expression points, determines the sound feature change amplitude according to the pitch feature change amplitude, the speech rate feature change amplitude and the volume feature change amplitude, and determines the physiological feature change amplitude according to the heart rate feature change amplitude, the blood pressure feature change amplitude and the skin galvanic response feature change amplitude; determines the facial feature weight, the sound feature weight and the physiological feature weight according to the facial feature change amplitude, the sound feature change amplitude and the physiological feature change amplitude; and fuses the facial feature emotion analysis result, the sound feature emotion analysis result and the physiological feature emotion analysis result according to the facial feature weight, the sound feature weight and the physiological feature weight to obtain the second emotional state information.

[0047] In the embodiment of the present application, optionally, determining the emotional state information of the target user according to the first emotional state information and the second emotional state information includes:

[0048] The feature analysis module compares and analyzes the first emotional state information and the second emotional state information, and generates the target user's emotional state recognition difference information based on the difference between the second emotional state information and the first emotional state information; if the emotional state recognition difference information is within a preset threshold, the average of the first emotional state information and the second emotional state information is determined as the emotional state information; if the emotional state recognition difference information is not within the preset threshold, the difference between the emotional state recognition difference information and the preset threshold is calculated, the ratio of the difference to the preset threshold is calculated, the difference between 0.5 and the ratio is used to determine the fusion weight of the first emotional state information, the sum of 0.5 and the ratio is determined as the fusion weight of the second emotional state information, and the first emotional state information and the second emotional state information are weightedly fused to determine the emotional state information.

[0049] In the embodiment of the present application, optionally, the method further includes:

[0050] The feedback module generates emotion change information according to the emotion state information and the emotion state information of the previous period corresponding to the emotion state information, and sends the emotion change information to the display device;

[0051] The display device displays the emotion feedback information and the emotion change information.

[0052] In the embodiment of the present application, optionally, the method further includes:

[0053] The data acquisition module acquires the audio data recorded during the online psychological consultation for the target user, and performs text conversion on the audio data to obtain the consultation dialogue data of the online psychological consultation;

[0054] The feedback module generates emotional fluctuation information of online psychological consultation based on the consultation dialogue data and the first emotional state information.

[0055] In the embodiment of the present application, optionally, the method further includes:

[0056] The feedback module divides the consultation conversation data into a plurality of conversation segment data according to different emotional state types based on the time correspondence between the first emotional state information and the consultation conversation data, and divides the conversation segment data into conversation segment data groups according to the emotional state types, analyzes the key information causing the change of the emotional state type of the target user based on the conversation segment data groups, and pushes the key information and the corresponding emotional state type to the terminal device of the psychological counselor.

[0057] In the embodiment of the present application, optionally, the method further includes:

[0058] The model training module constructs training samples according to the emotional state information and the physiological characteristics, and uses the training samples to train the physiological characteristic emotion recognition model corresponding to the target user;

[0059] The emotion monitoring module obtains the daily physiological signal data of the target user in daily life collected by the smart monitoring device worn by the target user, extracts the daily physiological characteristics corresponding to the daily physiological signal data, and uses the physiological characteristic emotion recognition model to identify the daily emotional state information of the target user according to the daily physiological characteristics.

[0060] In the embodiment of the present application, optionally, the method further includes:

[0061] The emotion monitoring module pushes a mental health resource link to the terminal device of the target user according to the daily emotion state information, and pushes the daily emotion state information to the terminal device of the psychological counselor.

[0062] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned psychological counseling assistance method based on user feature analysis is implemented.

[0063] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned psychological counseling assistance method based on user feature analysis when executing the program.

[0064] By means of the above technical solution, a psychological counseling auxiliary system and method based on user feature analysis is provided in the embodiment of the present application. In the process of online psychological counseling, the system collects facial images, voice data and physiological signal data of the target user through the data acquisition module. The feature analysis module then extracts the facial, voice and physiological features in these data, and analyzes the user's emotional state in real time. The feedback module generates emotional feedback information based on the analysis results, and sends it to the display device for reference by the psychological counselor, so as to provide psychological counseling services to the user more accurately. Through the comprehensive analysis of multi-source data, the embodiment of the present application can more comprehensively capture the changes in the user's emotional state, provide more accurate emotional feedback to the psychological counselor, and thus help the counselor to formulate a more personalized counseling plan. The real-time feedback mechanism enables the psychological counselor to adjust the counseling strategy in real time, avoid wasting time on invalid topics, and improve the overall efficiency of counseling. The user feels the counselor's accurate understanding and attention to his or her emotional state, which can enhance the trust and sense of security in the counseling process, thereby improving the counseling experience. For psychological counselors, with the help of this auxiliary system, they can learn and master emotion recognition and coping skills more systematically, and promote the improvement of personal professional ability.

[0065] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0067] Figure 1 A schematic diagram of the structure of a psychological counseling assistance system based on user feature analysis provided in an embodiment of the present application is shown;

[0068] Figure 2 A flow chart of a psychological counseling assistance method based on user feature analysis provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0069] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0070] In this embodiment, a psychological consultation auxiliary system based on user feature analysis is provided. Figure 1 As shown, including:

[0071] The data acquisition module 101 is used to obtain the facial image of the target user collected by a camera device, obtain the voice data of the target user collected by a sound collecting device, and obtain the physiological signal data of the target user collected by an intelligent monitoring device worn by the target user in response to the online psychological consultation start signal of the target user;

[0072] The feature analysis module 102 is used to extract the facial features corresponding to the facial image, the sound features corresponding to the voice data, and the physiological features corresponding to the physiological signal data, and analyze the emotional state information of the target user in real time based on the facial features, the sound features, and the physiological features;

[0073] A feedback module 103 is used to generate emotional feedback information according to the emotional state information of the target user, and send the emotional feedback information to the display device 104;

[0074] The display device 104 is used to display the emotional feedback information for reference by the psychological counselor so as to continue to provide psychological counseling services to the target user.

[0075] The embodiment of the present application provides a psychological counseling auxiliary system based on user feature analysis, which aims to provide an auxiliary tool for psychological counselors through multi-source data collection and real-time analysis, so as to more comprehensively understand the emotional state of the target user during the consultation process, thereby improving the effect and efficiency of psychological counseling. The system includes: a data acquisition module 101, a feature analysis module 102, a feedback module 103, and a display device 104. The data acquisition module 101 performs facial image acquisition, voice data acquisition, and physiological signal data acquisition. Among them, the facial expression changes of the target user are captured in real time by a camera device. Facial expression is one of the important ways to express emotions and can reflect the inner state of the user; the user's voice is recorded by a sound collection device, and the tone, speech speed, volume and other information in the voice can reflect the user's emotional state; the user's physiological signals, such as heart rate, blood pressure, skin conductance, etc., are collected by intelligent monitoring equipment (such as a heart rate monitor, a skin conductivity meter, etc.), and these physiological indicators are closely related to the emotional state. The feature analysis module 102 is responsible for extracting key features from the collected data, including facial features (such as the degree of eye opening and closing, the curvature of the corners of the mouth, etc.), voice features (such as pitch, speech speed, etc.) and physiological features (such as heart rate variability, skin conductance level, etc.). Based on these features, the current emotional state of the target user, such as anxiety, sadness, anger, calmness, etc., is analyzed and inferred in real time. The feedback module 103 generates emotional feedback information based on the results of the feature analysis module. This information may be presented in the form of charts, text descriptions or emotional tags. The emotional feedback information is sent to the display device 104 to ensure that the psychological counselor can see this information immediately. The display device 104 (such as a computer screen, a tablet, etc.) is used to intuitively display the emotional feedback information and provide an immediate reference for the psychological counselor. The psychological counselor can adjust the consulting strategy based on this information and respond to the needs of the target user more accurately. In addition, it should be noted that the various user data used in this application are collected with the permission of the user.

[0076] By applying the technical solution of this embodiment, during the online psychological consultation process, the system collects the facial image, voice data and physiological signal data of the target user through the data acquisition module. The feature analysis module then extracts the facial, voice and physiological features in these data, and analyzes the user's emotional state in real time. The feedback module generates emotional feedback information based on the analysis results, and sends it to the display device for reference by the psychological counselor, so as to provide psychological counseling services to the user more accurately. Through the comprehensive analysis of multi-source data, the embodiment of the present application can more comprehensively capture the changes in the user's emotional state, provide more accurate emotional feedback to the psychological counselor, and thus help the counselor to formulate a more personalized counseling plan. The real-time feedback mechanism enables the psychological counselor to adjust the counseling strategy in real time, avoid wasting time on invalid topics, and improve the overall efficiency of the consultation. The user feels the counselor's accurate understanding and attention to his or her emotional state, which can enhance the trust and sense of security in the consultation process, thereby improving the consultation experience. For psychological counselors, with the help of this auxiliary system, they can learn and master emotion recognition and coping skills more systematically, and promote the improvement of personal professional ability.

[0077] In an embodiment of the present application, optionally, the feature analysis module 102 is also used to: fuse the facial features, the voice features and the physiological features to obtain a user fusion feature, and perform real-time analysis of the emotional state of the target user using the user fusion feature to obtain first emotional state information; perform real-time analysis of the emotional state of the facial features, the voice features and the physiological features, respectively, and fuse the facial feature emotional analysis results, the voice feature emotional analysis results and the physiological feature emotional analysis results to obtain second emotional state information; determine the emotional state information of the target user based on the first emotional state information and the second emotional state information.

[0078] In this embodiment, the feature analysis module 102 first performs a fusion process on the facial features, the voice features and the physiological features to obtain a user fusion feature. Subsequently, the target user is subjected to a real-time analysis of the emotional state using the fusion feature to obtain the first emotional state information. For example, a multimodal feature emotion recognition model is used to perform a fusion process on the facial features, the voice features and the physiological features to obtain a user fusion feature, and the user fusion feature is subjected to emotion recognition. The multimodal feature emotion recognition model may include a multimodal feature fusion network and an emotion recognition network, and the multimodal feature emotion recognition model is trained using multimodal feature samples (including facial features, voice features and physiological features) and emotion information labels corresponding to the multimodal feature samples. At the same time, the feature analysis module 102 also performs independent real-time analysis of the emotional state of the facial features, the voice features and the physiological features. This analysis method helps to understand the independent role of each feature in the emotional expression and the degree of their respective contribution to the user's emotional state. The analysis results are the facial feature emotion analysis results, the voice feature emotion analysis results and the physiological feature emotion analysis results. These results may be presented in the form of probability distribution, emotion label or emotion score. Finally, the feature analysis module 102 fuses the sentiment analysis results of the above three single features to obtain the second sentiment state information, aiming to comprehensively consider the sentiment analysis results of different features to obtain a more comprehensive and accurate judgment of the user's sentiment state. Thus, based on the first sentiment state information (based on the analysis results of the user's fusion features) and the second sentiment state information (based on the fusion of the independent analysis results of a single feature), the feature analysis module 102 finally determines the target user's sentiment state information. For example, a weighted average of the two sentiment state information, a voting mechanism or a more complex decision logic is used to ensure the accuracy and reliability of the final result. By fusing feature information of multiple dimensions, the embodiment of the present application enables the system to more comprehensively capture the changes in the user's sentiment state and reduce misjudgments caused by the limitations of a single feature. Even in the case where some feature data is missing or of poor quality, the system can still infer the user's sentiment state through the analysis results of other features, thereby improving the stability and reliability of the system. The independent analysis of a single feature helps to understand the specific role of different features in emotional expression, while the fusion analysis can reveal the inherent connection and mutual influence between these features, thereby providing users with a deeper understanding of the sentiment state. Based on the emotional state analysis results of multi-feature fusion, psychological counselors can develop more personalized and targeted counseling strategies to meet the specific needs of different users.

[0079] In the embodiment of the present application, optionally, the facial features include position features and morphological features of facial expression feature points, the sound features include pitch features, speech speed features and volume features, and the physiological features include heart rate features, blood pressure features and skin electrical response features; the feature analysis module 102 is also used to: respectively compare the position features of the facial expression feature points, the morphological features of the facial expression feature points, the pitch features, the speech speed features, the volume features, the heart rate features, the blood pressure features and the skin electrical response features with the features of the previous time period corresponding to their respective feature types, and determine the feature change amplitude corresponding to each feature type; according to the change amplitude of the position features of the facial expression feature points and the facial The facial feature change amplitude is determined by the morphological feature change amplitude of the expression point, the sound feature change amplitude is determined according to the pitch feature change amplitude, the speech speed feature change amplitude and the volume feature change amplitude, and the physiological feature change amplitude is determined according to the heart rate feature change amplitude, the blood pressure feature change amplitude and the skin electrical response feature change amplitude; the facial feature weight, the sound feature weight and the physiological feature weight are determined according to the facial feature change amplitude, the sound feature change amplitude and the physiological feature change amplitude; according to the facial feature weight, the sound feature weight and the physiological feature weight, the facial feature emotion analysis result, the sound feature emotion analysis result and the physiological feature emotion analysis result are fused to obtain the second emotional state information.

[0080] In this embodiment, facial features include positional features and morphological features of facial expression feature points. These feature points may include specific points of key parts such as eyebrows, eyes, and mouths, and their position and morphological changes can reflect changes in facial expressions. Voice features include pitch features, speech rate features, and volume features. These features can reflect the emotional state of the user when speaking, such as the pitch may indicate excitement or frustration, the speed of speech may indicate tension or relaxation, and the change in volume may be related to the intensity of emotions. Physiological features include heart rate features, blood pressure features, and skin galvanic response features. These physiological indicators can reflect the user's autonomic nervous system activities and are closely related to the emotional state. The feature analysis module 102 will compare the positional features, morphological features, pitch features, speech rate features, volume features, and heart rate features, blood pressure features, and skin galvanic response features of the facial expression feature points of the current time period with the features of the previous time period corresponding to each feature type. By calculating the amount of change of these features between the previous and next time periods, the feature change amplitude corresponding to each feature type can be determined. The previous time period may refer to various features within the previous 5 seconds, 3 seconds, or other durations. The feature analysis module 102 further determines the weights of facial features, voice features and physiological features according to the feature change amplitude of each feature type. The determination of these weights can be based on the following principles: feature types with large change amplitudes may be more important: because they have changed significantly in a short period of time, they are more likely to reflect the rapid change of the user's emotional state. Support of historical data or expert knowledge: Some feature types may have higher importance in emotion recognition, which can be obtained by analyzing historical data or consulting expert opinions. After determining the weights of each feature type, the feature analysis module 102 will fuse the facial feature emotion analysis results, the voice feature emotion analysis results and the physiological feature emotion analysis results according to these weights. The fusion process may involve weighted summation, weighted average or other complex fusion algorithms, which aims to comprehensively consider the emotion analysis results of different feature types to obtain more comprehensive and accurate second emotional state information. In the embodiment of the present application, by comparing the feature change amplitudes and determining the feature weights, the system can more accurately capture the changes in the user's emotional state and perform emotion recognition accordingly. The determination of the weights takes into account the change amplitude of the features, so that the system can make adaptive adjustments according to the changes in the importance of the features in different situations, thereby improving the robustness of the system. By integrating the sentiment analysis results of multiple feature types, the system can provide a more refined description of the emotional state, helping psychological counselors to understand the user's emotional needs more deeply.

[0081] In an embodiment of the present application, optionally, the feature analysis module 102 is also used to: perform comparative analysis on the first emotional state information and the second emotional state information, and generate emotional state recognition difference information of the target user based on the difference between the second emotional state information and the first emotional state information; if the emotional state recognition difference information is within a preset threshold, then determine the average of the first emotional state information and the second emotional state information as the emotional state information; if the emotional state recognition difference information is not within the preset threshold, calculate the difference between the emotional state recognition difference information and the preset threshold, calculate the ratio of the difference to the preset threshold, determine the fusion weight of the first emotional state information by the difference between 0.5 and the ratio, determine the sum of 0.5 and the ratio as the fusion weight of the second emotional state information, and perform weighted fusion of the first emotional state information and the second emotional state information to determine the emotional state information.

[0082] In this embodiment, the feature analysis module 102 first performs a comparative analysis on the first emotional state information and the second emotional state information. The first emotional state information is obtained by performing a real-time emotional state analysis based on the user's fusion features (fusion of facial features, voice features and physiological features), while the second emotional state information is obtained by fusion of the emotional analysis results of each single feature (facial features, voice features and physiological features). Through comparative analysis, the module generates emotional state recognition difference information of the target user, which reflects the degree of difference between the two emotional state information. Next, the module determines how to fuse the first emotional state information and the second emotional state information according to the emotional state recognition difference information to obtain the final emotional state information. If the emotional state recognition difference information is within the preset threshold, it means that the difference between the two emotional state information is small, which may be due to the redundancy or consistency of different feature types when expressing emotions. At this time, the module determines the mean of the first emotional state information and the second emotional state information as the final emotional state information. This strategy is simple and effective, and can balance the influence of different feature types on the emotional state. If the emotional state recognition difference information is not within the preset threshold, it means that the difference between the two emotional state information is large, which may be due to the large difference or conflict between different feature types when expressing emotions. At this time, the module will further calculate the difference between the emotional state recognition difference information and the preset threshold, and calculate the ratio of the difference to the preset threshold. Then, this ratio is used to dynamically adjust the fusion weight of the first emotional state information and the second emotional state information. Specifically, the difference between 0.5 (i.e., the initial equal weight distribution) and the ratio is used as the fusion weight of the first emotional state information, and the sum of 0.5 and the ratio is used as the fusion weight of the second emotional state information. In this way, when the difference information is far away from the preset threshold, the weight will be inclined to more reliable or consistent emotional state information, thereby obtaining more accurate and stable final emotional state information. The embodiment of the present application compares and analyzes the differences between different emotional state information, and dynamically adjusts the fusion weight according to the degree of difference, so that the system can more accurately reflect the user's true emotional state and improve the accuracy and stability of emotion recognition. The fusion strategy can be adaptively adjusted according to the changes in the importance of feature types in different situations, so that the system can maintain good performance in different scenarios, and enhance the adaptability and robustness of the system. By considering the differences and consistency of different feature types in emotional expression, the system can provide a more refined description and analysis of emotional state, which helps psychological counselors to have a deeper understanding of the user's emotional needs and changes.

[0083] In an embodiment of the present application, optionally, the feedback module 103 is also used to generate emotion change information based on the emotion state information and the emotion state information of a previous time period corresponding to the emotion state information, and send the emotion change information to the display device 104; the display device 104 is also used to display the emotion feedback information and the emotion change information.

[0084] In this embodiment, the feedback module 103 is not only responsible for processing the feedback of the emotional state information, but also can generate the function of emotional change information according to the comparison between the current emotional state information and the emotional state information of the previous period. This function is crucial to understanding the dynamic changes of the user's emotions, because it can reveal the evolution trend of the user's emotional state over time. Specifically, the feedback module 103 will first obtain the current emotional state information and the emotional state information of the previous period stored in the system. Then, by comparing and analyzing the emotional state information of these two periods, the module will generate emotional change information. This change information may include changes in the intensity of the emotional state, type changes (such as from happiness to sadness), etc. The display device 104 can also display the function of emotional change information on the basis of the original display of emotional feedback information. This means that when the user's emotional state changes, the system can not only feedback the current emotional state in real time, but also intuitively display this change through the display device 104, so that the user or the psychological counselor can see the dynamic evolution process of the emotional state more clearly. By displaying the emotional change information, the embodiment of the present application can more accurately evaluate the changes in the user's emotional state according to the emotional change information, and formulate a more personalized and effective psychological counseling plan.

[0085] In an embodiment of the present application, optionally, the data acquisition module 101 is also used to obtain audio data recorded during the online psychological consultation process of the target user, and to convert the audio data into text to obtain consultation dialogue data of the online psychological consultation; the feedback module 103 is also used to generate emotional fluctuation information of the online psychological consultation based on the consultation dialogue data and the first emotional state information.

[0086] In this embodiment, the data acquisition module 101 can also acquire the function of recording data recorded by the target user during the online psychological consultation process. After acquiring the recording data, the data acquisition module 101 will also use text conversion technology (such as speech recognition technology) to process the recording data and convert it into a readable text format, that is, the consultation dialogue data of the online psychological consultation. The feedback module 103 has the function of generating the emotional fluctuation information of the online psychological consultation based on the consultation dialogue data and the first emotional state information. This function can more comprehensively reflect the emotional changes of the user during the consultation process by combining text data with emotional state information. Specifically, the feedback module 103 will first analyze the keywords, phrases or sentences in the consultation dialogue data to identify the content related to emotion. Then, these contents are associated and compared with the first emotional state information (real-time emotional state information obtained based on the fusion feature), so as to infer the emotional fluctuation of the user during the consultation process. Finally, the system will generate emotional fluctuation information, which may include the changing trend of the emotional state, the occurrence time of emotional peaks and troughs, and the corresponding consultation content. By combining the consultation dialogue data and emotional state information, the system can more comprehensively understand the user's emotional state and emotional fluctuations during online psychological consultation. Psychological counselors can more accurately grasp the user's emotional changes and consultation needs based on the emotional fluctuation information, thereby formulating more personalized and effective psychological consultation strategies.

[0087] In an embodiment of the present application, optionally, the feedback module 103 is also used to divide the consultation conversation data into a plurality of conversation segment data according to different emotional state types based on the time correspondence between the first emotional state information and the consultation conversation data, and to divide the conversation segment data into conversation segment data groups according to the emotional state types, analyze the key information that causes the change in the emotional state type of the target user based on the conversation segment data groups, and push the key information and the corresponding emotional state type to the terminal device of the psychological counselor.

[0088] In this embodiment, the feedback module 103 accurately divides the consultation dialogue data into a plurality of dialogue segment data according to the time correspondence between the first emotional state information and the consultation dialogue data. Each dialogue segment data corresponds to a specific stage or change point of the user's emotional state. Then, the dialogue segment data is further divided into different dialogue segment data groups according to the emotional state type. In this way, dialogue segments with similar emotional state types will be classified together for subsequent analysis and processing. Based on the dialogue segment data groups, the feedback module 103 will deeply analyze the dialogue content in each group to identify the key information that causes the target user's emotional state type to change. These key information may be specific speech, topic or emotional trigger point, which has a significant impact on the user's emotional state. Finally, the feedback module 103 pushes the relevant information to the terminal device of the psychological counselor according to the key information obtained by the analysis and the corresponding emotional state type. In this way, the psychological counselor can instantly obtain the key information of the user's emotional state change during the consultation process, so as to more accurately understand the user's emotional needs and reasons for change, and then provide more accurate and effective psychological counseling services. By deeply analyzing the relationship between consultation dialogue data and changes in emotional state, the embodiment of the present application can accurately identify the key factors that affect the user's emotional state and provide more accurate consultation guidance for psychological counselors.

[0089] In an embodiment of the present application, optionally, it also includes: a model training module 105, which is used to construct training samples based on the emotional state information and the physiological characteristics, and use the training samples to train the physiological characteristic emotion recognition model corresponding to the target user; an emotion monitoring module 106, which is used to obtain daily physiological signal data of the target user in daily life collected by the smart monitoring device worn by the target user, extract daily physiological characteristics corresponding to the daily physiological signal data, and use the physiological characteristic emotion recognition model to identify the daily emotional state information of the target user according to the daily physiological characteristics.

[0090] In this embodiment, the model training module 105 constructs a training sample set based on the existing emotional state information and the corresponding physiological characteristic data (such as heart rate, skin conductivity, breathing rate, etc.). Using the constructed training sample set, a model that can accurately identify the relationship between physiological characteristics and emotional state is trained through machine learning or deep learning algorithms. This model will learn the specific patterns of physiological characteristics under different emotional states, so that it can predict the user's emotional state based on physiological characteristic data in the future. After the model training is completed, the emotion monitoring module 106 can use this physiological characteristic emotion recognition model to monitor the target user's daily emotional state in real time. The specific process is as follows: Through the smart monitoring device (such as a smart watch, a health bracelet, etc.) worn by the target user, the user's physiological signal data in daily life, such as heart rate, blood pressure, skin conductivity, etc., is continuously collected. The collected daily physiological signal data is preprocessed and feature extracted to obtain data corresponding to the physiological characteristics used in model training. Using the trained physiological characteristic emotion recognition model, the target user's daily emotional state is identified in real time based on the extracted daily physiological characteristics. This process is automated and can be performed without the user's awareness, providing users with continuous, real-time emotion monitoring services. By constructing and training a physiological characteristic emotion recognition model for a specific user, the embodiment of the present application can more accurately identify the user's emotional state and reduce misjudgments and missed judgments. The relationship between each user's physiological characteristics and emotional state may be different. By training a special emotion recognition model for each user, the user's emotional state can be monitored more personalized, improving the pertinence and effectiveness of monitoring. Through the combination of intelligent monitoring equipment and physiological characteristic emotion recognition models, continuous and real-time monitoring of the user's daily emotional state can be achieved, providing users with more comprehensive and detailed emotional support services.

[0091] In an embodiment of the present application, optionally, the emotion monitoring module 106 is also used to push a mental health resource link to the terminal device of the target user based on the daily emotional state information, and to push the daily emotional state information to the terminal device of the psychological counselor.

[0092] In this embodiment, after identifying the target user's daily emotional state information, the emotion monitoring module 106 will intelligently select and push relevant mental health resource links to the user's terminal device according to the user's emotional state and needs. These resources may include mental health articles, video tutorials, relaxing music, meditation guidance, contact information of professional psychological counseling agencies, etc., which are intended to help users understand their own emotional state, learn how to regulate emotions, and seek professional psychological help when needed. At the same time, the emotion monitoring module 106 will also push the target user's daily emotional state information to the terminal device of the psychological counselor in real time or regularly. In this way, the psychological counselor can understand the user's emotional changes in a timely manner, evaluate the user's mental health status, and actively contact the user when necessary to provide personalized consultation and support. This two-way information transmission mechanism helps to enhance the interaction and communication between the psychological counselor and the user, and improve the effectiveness and pertinence of mental health services. By pushing mental health resource links to users, the embodiment of the present application can guide users to pay attention to their own emotional state, learn how to regulate emotions, improve self-management ability, and promote mental health. Pushing daily emotional state information to psychological counselors helps strengthen the connection between counselors and users, enabling counselors to more comprehensively understand users' emotional states and needs and provide more accurate and personalized consulting services. By real-time monitoring and pushing emotional state information, it is possible to promptly identify possible psychological problems or crisis situations of users, and quickly take corresponding intervention measures to improve the timeliness and effectiveness of mental health services.

[0093] Furthermore, the embodiment of the present application provides a psychological counseling assistance method based on user feature analysis, which is applied to the above-mentioned psychological counseling assistance system based on user feature analysis, such as Figure 2 As shown, the method includes:

[0094] Step 201, in response to a target user's online psychological counseling start signal, the data acquisition module acquires a facial image of the target user acquired by a camera device, acquires voice data of the target user acquired by a sound collecting device, and acquires physiological signal data of the target user acquired by a smart monitoring device worn by the target user;

[0095] Step 202: The feature analysis module extracts the facial features corresponding to the facial image, the sound features corresponding to the voice data, and the physiological features corresponding to the physiological signal data, and analyzes the emotional state information of the target user in real time based on the facial features, the sound features, and the physiological features;

[0096] Step 203: the feedback module generates emotional feedback information according to the emotional state information of the target user, and sends the emotional feedback information to a display device;

[0097] Step 204: the display device displays the emotional feedback information for reference by the psychological counselor before continuing to provide psychological counseling services to the target user.

[0098] In an embodiment of the present application, optionally, analyzing the target user's emotional state information in real time according to the facial features, the voice features, and the physiological features includes:

[0099] The feature analysis module fuses the facial features, the voice features and the physiological features to obtain a user fusion feature, and performs real-time analysis of the emotional state of the target user using the user fusion feature to obtain first emotional state information; performs real-time analysis of the emotional state of the facial features, the voice features and the physiological features respectively, and fuses the facial feature emotional analysis results, the voice feature emotional analysis results and the physiological feature emotional analysis results to obtain second emotional state information; determines the emotional state information of the target user based on the first emotional state information and the second emotional state information.

[0100] In the embodiment of the present application, optionally, the facial features include position features and morphological features of facial expression feature points, the voice features include pitch features, speech speed features and volume features, and the physiological features include heart rate features, blood pressure features and skin electrical response features;

[0101] The facial feature emotion analysis results, the voice feature emotion analysis results, and the physiological feature emotion analysis results are integrated to obtain the second emotion state information, including:

[0102] The feature analysis module compares the position features of the facial expression feature points, the morphological features of the facial expression feature points, the pitch features, the speech rate features, the volume features, the heart rate features, the blood pressure features and the skin galvanic response features with the features of the previous time period corresponding to the respective feature types to determine the feature change amplitude corresponding to each feature type; determines the facial feature change amplitude according to the position feature change amplitude of the facial expression feature points and the morphological feature change amplitude of the facial expression points, determines the sound feature change amplitude according to the pitch feature change amplitude, the speech rate feature change amplitude and the volume feature change amplitude, and determines the physiological feature change amplitude according to the heart rate feature change amplitude, the blood pressure feature change amplitude and the skin galvanic response feature change amplitude; determines the facial feature weight, the sound feature weight and the physiological feature weight according to the facial feature change amplitude, the sound feature change amplitude and the physiological feature change amplitude; and fuses the facial feature emotion analysis result, the sound feature emotion analysis result and the physiological feature emotion analysis result according to the facial feature weight, the sound feature weight and the physiological feature weight to obtain the second emotional state information.

[0103] In the embodiment of the present application, optionally, determining the emotional state information of the target user according to the first emotional state information and the second emotional state information includes:

[0104] The feature analysis module compares and analyzes the first emotional state information and the second emotional state information, and generates the target user's emotional state recognition difference information based on the difference between the second emotional state information and the first emotional state information; if the emotional state recognition difference information is within a preset threshold, the average of the first emotional state information and the second emotional state information is determined as the emotional state information; if the emotional state recognition difference information is not within the preset threshold, the difference between the emotional state recognition difference information and the preset threshold is calculated, the ratio of the difference to the preset threshold is calculated, the difference between 0.5 and the ratio is used to determine the fusion weight of the first emotional state information, the sum of 0.5 and the ratio is determined as the fusion weight of the second emotional state information, and the first emotional state information and the second emotional state information are weightedly fused to determine the emotional state information.

[0105] In the embodiment of the present application, optionally, the method further includes:

[0106] The feedback module generates emotion change information according to the emotion state information and the emotion state information of the previous period corresponding to the emotion state information, and sends the emotion change information to the display device;

[0107] The display device displays the emotion feedback information and the emotion change information.

[0108] In the embodiment of the present application, optionally, the method further includes:

[0109] The data acquisition module acquires the audio data recorded during the online psychological consultation for the target user, and performs text conversion on the audio data to obtain the consultation dialogue data of the online psychological consultation;

[0110] The feedback module generates emotional fluctuation information of online psychological consultation based on the consultation dialogue data and the first emotional state information.

[0111] In the embodiment of the present application, optionally, the method further includes:

[0112] The feedback module divides the consultation conversation data into a plurality of conversation segment data according to different emotional state types based on the time correspondence between the first emotional state information and the consultation conversation data, and divides the conversation segment data into conversation segment data groups according to the emotional state types, analyzes the key information causing the change of the emotional state type of the target user based on the conversation segment data groups, and pushes the key information and the corresponding emotional state type to the terminal device of the psychological counselor.

[0113] In the embodiment of the present application, optionally, the method further includes:

[0114] The model training module constructs training samples according to the emotional state information and the physiological characteristics, and uses the training samples to train the physiological characteristic emotion recognition model corresponding to the target user;

[0115] The emotion monitoring module obtains the daily physiological signal data of the target user in daily life collected by the smart monitoring device worn by the target user, extracts the daily physiological characteristics corresponding to the daily physiological signal data, and uses the physiological characteristic emotion recognition model to identify the daily emotional state information of the target user according to the daily physiological characteristics.

[0116] In the embodiment of the present application, optionally, the method further includes:

[0117] The emotion monitoring module pushes a mental health resource link to the terminal device of the target user according to the daily emotion state information, and pushes the daily emotion state information to the terminal device of the psychological counselor.

[0118] The embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a bus, a processor, a memory and a communication interface, and can also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0119] Those skilled in the art will appreciate that the structure of the above-mentioned computer device is only a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0120] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0121] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0124] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A psychological counseling assistance system based on user feature analysis, characterized in that: include: A data acquisition module, for responding to a target user's online psychological counseling start signal, acquiring a facial image of the target user acquired by a camera device, acquiring voice data of the target user acquired by a sound collecting device, and acquiring physiological signal data of the target user acquired by an intelligent monitoring device worn by the target user; A feature analysis module, used to extract facial features corresponding to the facial image, sound features corresponding to the voice data, and physiological features corresponding to the physiological signal data, and analyze the emotional state information of the target user in real time based on the facial features, the sound features, and the physiological features; A feedback module, used to generate emotional feedback information according to the emotional state information of the target user, and send the emotional feedback information to a display device; The display device is used to display the emotional feedback information for reference by the psychological counselor so as to continue to provide psychological counseling services to the target user.

2. The psychological counseling assistance system according to claim 1, characterized in that: The feature analysis module is also used for: The facial features, the voice features and the physiological features are fused to obtain a user fusion feature, and the emotional state of the target user is analyzed in real time using the user fusion feature to obtain first emotional state information; Performing real-time emotional state analysis on the facial features, the voice features, and the physiological features respectively, and fusing the facial feature emotional analysis results, the voice feature emotional analysis results, and the physiological feature emotional analysis results to obtain second emotional state information; The emotional state information of the target user is determined according to the first emotional state information and the second emotional state information.

3. The psychological counseling assistance system according to claim 2, characterized in that: The facial features include position features and morphological features of facial expression feature points, the voice features include pitch features, speech speed features and volume features, and the physiological features include heart rate features, blood pressure features and skin electrical response features; The feature analysis module is further used for: Respectively comparing the position features of the facial expression feature points, the morphological features of the facial expression feature points, the tone features, the speech speed features, the volume features, the heart rate features, the blood pressure features, and the skin electrical response features with the features of the previous time period corresponding to the respective feature types, and determining the feature change amplitude corresponding to each feature type; Determine the facial feature change amplitude according to the position feature change amplitude of the facial expression feature point and the morphological feature change amplitude of the facial expression point, determine the sound feature change amplitude according to the pitch feature change amplitude, the speech speed feature change amplitude and the volume feature change amplitude, and determine the physiological feature change amplitude according to the heart rate feature change amplitude, the blood pressure feature change amplitude and the skin electrical response feature change amplitude; Determining facial feature weights, voice feature weights, and physiological feature weights according to the facial feature change range, the voice feature change range, and the physiological feature change range; According to the facial feature weights, the voice feature weights and the physiological feature weights, the facial feature emotion analysis results, the voice feature emotion analysis results and the physiological feature emotion analysis results are fused to obtain the second emotion state information.

4. The psychological counseling assistance system according to claim 3, characterized in that: The feature analysis module is also used for: Comparing and analyzing the first emotional state information and the second emotional state information, and generating emotional state recognition difference information of the target user according to the difference between the second emotional state information and the first emotional state information; If the emotional state recognition difference information is within a preset threshold, determining the average of the first emotional state information and the second emotional state information as the emotional state information; If the emotional state recognition difference information is not within the preset threshold, the difference between the emotional state recognition difference information and the preset threshold is calculated, the ratio of the difference to the preset threshold is calculated, the difference between 0.5 and the ratio is used to determine the fusion weight of the first emotional state information, the sum of 0.5 and the ratio is determined as the fusion weight of the second emotional state information, and the first emotional state information and the second emotional state information are weightedly fused to determine the emotional state information.

5. The psychological counseling assistance system according to claim 4, characterized in that: The feedback module is further configured to generate emotion change information according to the emotion state information and the emotion state information of a previous period corresponding to the emotion state information, and send the emotion change information to the display device; The display device is also used to display the emotion feedback information and the emotion change information.

6. The psychological counseling assistance system according to claim 4, characterized in that: The data acquisition module is also used to obtain the audio data recorded during the online psychological consultation process for the target user, and to convert the audio data into text to obtain the consultation dialogue data of the online psychological consultation; The feedback module is also used to generate emotional fluctuation information of online psychological consultation based on the consultation dialogue data and the first emotional state information.

7. The psychological counseling assistance system according to claim 6, characterized in that: The feedback module is also used to divide the consultation conversation data into multiple conversation segment data according to different emotional state types based on the time correspondence between the first emotional state information and the consultation conversation data, and divide the conversation segment data into conversation segment data groups according to the emotional state types, analyze the key information that causes the change of the emotional state type of the target user based on the conversation segment data groups, and push it to the terminal device of the psychological counselor according to the key information and the corresponding emotional state type.

8. The psychological counseling assistance system according to claim 4, characterized in that: Also includes: A model training module, used to construct training samples according to the emotional state information and the physiological characteristics, and use the training samples to train the physiological characteristic emotion recognition model corresponding to the target user; The emotion monitoring module is used to obtain the daily physiological signal data of the target user in daily life collected by the smart monitoring device worn by the target user, extract the daily physiological characteristics corresponding to the daily physiological signal data, and use the physiological characteristic emotion recognition model to identify the daily emotional state information of the target user according to the daily physiological characteristics.

9. The psychological counseling assistance system according to claim 8, characterized in that: The emotion monitoring module is also used to push a mental health resource link to the terminal device of the target user based on the daily emotion state information, and to push the daily emotion state information to the terminal device of the psychological counselor.

10. A psychological consultation assistance method based on user feature analysis, characterized in that: The psychological counseling auxiliary system based on user feature analysis as described in any one of claims 1 to 9 comprises: The data acquisition module responds to the target user's online psychological consultation start signal to acquire the target user's facial image acquired by a camera device, acquire the target user's voice data acquired by a sound collection device, and acquire the target user's physiological signal data acquired by a smart monitoring device worn by the target user; The feature analysis module extracts facial features corresponding to the facial image, sound features corresponding to the voice data, and physiological features corresponding to the physiological signal data, and analyzes the emotional state information of the target user in real time based on the facial features, the sound features, and the physiological features; The feedback module generates emotional feedback information according to the emotional state information of the target user, and sends the emotional feedback information to the display device; The display device displays the emotional feedback information for reference by the psychological counselor who then continues to provide psychological counseling services to the target user.

Images